Key Responsibilities Design, build, and maintain ETL/ELT pipelines in Databricks to process large-scale structured and unstructured datasets. Ensure timely ingestion, transformation, and delivery of data to internal consumers, analytics platforms, and machine learning models. Perform large-scale data transformation, cleaning, and aggregation using Spark and Databricks notebooks. Optimize data processing workflows for efficiency, scalability, and cost-effectiveness by fine-tuning Spark jobs and cluster configurations. The Ideal Candidate Qualifications: Bachelor's degree in Computer Science, Information Technology, or a related field. A Master’s degree is preferred. 6+ years of experience in data engineering, with projects in Databricks and Apache Spark. Strong hands-on experience with Databricks, including managing clusters, developing pipelines, and optimizing workloads. Proficiency in Python, Scala, or SQL for data processing within Databricks. Expertise in cloud platforms such as AWS, Azure, or GCP and their data services (e.g., S3, ADLS, BigQuery). Deep understanding of ETL/ELT processes, data modeling, and working with large-scale distributed systems. Familiarity with DevOps practices, including CI/CD, containerization, and infrastructure automation (e.g., Jenkins, Docker, Kubernetes). Experience with data governance, security, and compliance best practices. Excellent problem-solving skills and a proactive approach to identifying and resolving performance bottlenecks. Strong communication and interpersonal skills, with experience collaborating with both technical and non-technical stakeholders
Job Type: Full-time
Pay: $54.72 - $65.90 per hour
Expected hours: 40 per week
Benefits:
• 401(k)
• Flexible schedule
• Health insurance
• Life insurance
• Paid time off
Experience:
• ETL: 3 years (Required)
• Databricks: 2 years (Required)
Ability to Commute:
• Cincinnati, OH 45202 (Required)
Ability to Relocate:
• Cincinnati, OH 45202: Relocate before starting work (Required)
Work Location: In person
Location: Cincinnati, OH
Posted: Sept. 19, 2024, 11:34 p.m.
Apply Now Company Website