Job Summary
We are seeking a highly skilled Data Engineer to join our team in a hybrid role from either Plano or Chicago. The ideal candidate will have expertise in data pipeline engineering and optimization, with strong experience in Scala, PySpark, and AWS (specifically Lambdas and EMR). The Data Engineer will focus on building and optimizing scalable data pipelines and working with a range of cloud-based technologies.
Key Responsibilities
• Develop, optimize, and maintain scalable data pipelines using Scala and PySpark.
• Design and implement efficient data pipeline architectures in AWS, specifically utilizing Lambdas and EMR.
• Collaborate with cross-functional teams to understand data requirements and deliver optimized data solutions.
• Leverage SQL for querying and manipulating large datasets.
• Ensure data quality and performance through regular monitoring and optimization of pipelines.
Required Qualifications
• Strong experience with Scala and PySpark for data pipeline engineering.
• Expertise in AWS, specifically Lambdas and EMR.
• Proficient in SQL for data querying and optimization.
• Experience with data engineering and pipeline optimization.
Primary Skills
• Scala
• PySpark
• AWS - EMR and Lambdas
• Data Engineering
Education: Bachelors Degree
Location: Plano, TX
Posted: Oct. 8, 2024, 7:49 p.m.
Apply Now Company Website