Job Title: Spark/Scala Developer
Location: Brooklyn, OH
Job Function: Technology
Job Summary
We are seeking a skilled Spark/Scala Developer with proven experience in big data technologies. The ideal candidate will have a strong background in Scala, Spark, and SQL, with hands-on experience in building and managing data processing pipelines in a production environment.
Key Responsibilities
• Big Data Expertise: Apply advanced knowledge of big data technologies such as Spark, Hadoop, or Elasticsearch. Demonstrate experience in either implementation or data science.
• Scala and SQL Development: Write efficient code in Scala and advanced SQL queries. Utilize Hive and Parquet for data processing.
• GCP Data Lake: Leverage experience working with Google Cloud Platform (GCP) data lakes.
• Agile Environment: Work effectively within an Agile framework, contributing to iterative development and continuous improvement.
• Coding Skills: Use your strong coding background in Java, Python, or Scala to develop and maintain robust data solutions.
• Testing: Utilize testing libraries such as Scala Test or equivalent for other languages. Understand and explain various test types, including unit and integration tests, and automate testing processes.
• Data Pipelines: Build and maintain data processing pipelines for production environments, including traditional ETL and analytics pipelines.
Desired Skills
• Expertise in Scala and Spark with practical experience in big data technologies.
• Advanced SQL skills and experience with data formats such as Parquet.
• Experience with GCP data lakes and Agile methodologies.
• Proficiency in at least one major programming language (Java, Python, Scala).
• Familiarity with testing libraries and concepts, including unit and integration testing.
• Experience in developing and managing data pipelines for production environments.
Qualifications
• Bachelor’s Degree in Computer Science or a related field.
Skills: scala,spark,data,pipelines,testing,big data,sql,agile,data processing
Location: New York, NY
Posted: Sept. 2, 2024, 8:14 a.m.
Apply Now Company Website