• Leading Financial Institution
• Data lake and data warehousing SME
About Our Client
We have an exciting opportunity for a Data Engineering Lead to join our Data Centre of Excellence.
As Data Engineering Lead you'll play a key role as a data lake and data warehousing SME having advance expertise in the development of an AWS (Cloud) data lake architecture. You'll manage a small team of engineers, analysts, and designers, and deliver change across a portfolio of programmes within the engineering function. When managing this team, you'll be responsible for the full technology life cycle including the creation of required reference architectures, designs and delivery best practices and procedures.
If you're someone who's performed a similar role already, this is the perfect opportunity to develop your career.
Job Description
• Leading data engineering teams to design data pipelines that centre on maturing of develops approach and increasing our levels of automation
• Designing robust engineering and modern Devops solutions for the analytical data lake
• Ensuring enterprise data is catalogued in accordance with data governance standards through design, development and maintaining data solutions leveraging the power and flexibility of the data lake
• Establishing a delivery strategy which leverages the data platforms efficiently across the technology stack to ensure reuse and minimal duplication. To create golden source repositories that support business units and propositions alike
• Implementing solutions to prevent repetition of known issues/ risks and establish knowledge bases of lessons learned
• Working closely with enterprise and data architecture to ensure individual solutions fit into the overall group architecture
The Successful Applicant
• Expertise in cloud architecture ideally AWS services such as Athena, Glue, Lambda, S3, Amazon EMR and SageMaker
• Proven experience in SQL and Non-SQL database technology relevant to the implementation of a data warehouse or data lake
• Experience implementing data ingestion, processing, transformation, and delivery using various methods such as batch, streaming, and real-time
• Knowledgeable with DevOps tools and methodologies such as Git, Jenkins, Docker, or Kubernetes
• Knowledge of ETL tools and frameworks such as Apache Spark, Airflow, AWS IoT Analytics, IoT Device Management, and AWS IoT Events
• Proficient in programming languages such as Python, Java, Scala, SQL, Spark, and Unix shell scripting, including AWS CLI
• * Experience with data pipelining, distributed data processing, software engineering components, and data modelling concepts (Spark)
• A degree or masters Computer Science or a related field, or equivalent work experience
• Desirable cloud certification. For example, AWS Certified Data Associate or AWS Certified Solutions Architect- Associate certification
• Experience in using Agile methodologies (Scrum, Kanban, etc.)
What's on Offer
We offer competitive remuneration package and comprehensive fringe benefits including medical and life insurance, excellent learning & development opportunities and flexibility to the right candidate.
Location:
Posted: Aug. 13, 2024, 12:47 a.m.
Apply Now Company Website