Lead Big-Data Engineer – Cloud Engineering Platform Team
We are looking for a Lead Data Engineer to join the CTO Developer Experiences Platform Team to lead the implementation of the vision to build Data Lake House & MLOps on the AWS cloud.
On top of that the platform engineers are helping the product innovation team to build Generative AI experiences across products and services. You have an amazing opportunity to be part of the Generative AI revolution and build Generative AI products and services.
What are some of the things you do on a day-to-day basis?
● Plan and build Data Lake House in the AWS cloud based on future technologies most companies will use in only 3-5 years.
● Have end to end ownership: Design, develop, deploy, measure and maintain our data platform.
● Solve production issues in the framework code.
● Collaborate with 40+ data engineers/scientists, understand their functional needs and wants and build products to accelerate their productivity and quality of work.
● Research and build Generative AI services and applications.
● Lead and mentor data engineers, providing guidance and fostering a culture of continuous learning and professional development.
● Participate in the software engineering, processes and standards with an impact on the whole delivery organization.
● Communicate complex architectural concepts and recommendations effectively to both technical and non-technical stakeholders.
● Help the in analyzing and adopting new technologies including assessing and embedding startup technologies.
Requirements
● Previous experience in data engineering and familiarity with AWS databases services like RDS, SPARK, Glue, OpenSearch and more.
● 5+ years experience developing large scale distributed data systems and services, experience in Big Data tools.
● 5+ years of working experience with AWS Cloud and services.
● 5+ years of expertise in the Software design and architecture process.
● Experience with SQL and No-SQL.
● Has at least 3 + years of software or data engineering development experience in one or more of the following programming languages: Python, Java, or Scala.
● Production systems understanding.
● Has good understanding and experience of CI/CD practices and Git, including MLOps practices.
