Role Location Auckland or Wellington

Role Type: Permanent, Fulltime

Hours: 37.5 hours per week

As a Lead Engineer, you are expected to focus on best practice approaches in data engineering and development. You will provide specialist technical expertise, supports design and development of solutions that meet long term business and customer needs.

Your solutions should be stable, secure, scalable, extensible and maintainable. Your role includes ensuring that designs are in compliance with architectural standards and guidelines, advising on functional issues and mentoring developers as they work through designing, coding, configuring and testing solutions.

In this role you will have the opportunity to:

• Work closely with Architecture team to ensure solutions align with architectural strategy
• Lead the design, development, and optimisation of our composable data services, leveraging technologies such as Apache Airflow, Apache Spark and Python-based frameworks.
• Design and implement scalable and memory-efficient data pipelines and ETL/ELT processes.
• Champion best practices for data engineering, including coding standards, testing, monitoring, and operational excellence.
• Evaluate and introduce new tools and technologies to enhance our data platform capabilities.
• Provide technical guidance and mentorship to other data engineers, fostering a culture of learning and innovation.
• Collaborate closely with data engineers, analytics engineers, data scientists, and other stakeholders to understand data requirements and deliver robust solutions.
• Define and enforce data governance, security, and compliance standards within the platform.
• Troubleshoot and resolve complex technical issues related to the data infrastructure.
• Contribute to the strategic planning and roadmap development for the data platform, aligning with overall business objectives.
• Focus on optimising Python and Airflow pipeline processes for memory efficiency to ensure cost-effective and performant operations.
• Be part of an awesome team as well as being well-supported with opportunities for your personal development and growth.

Along with your experience as a Lead Engineer, combined functional and technical knowledge you will be a skilled communicator comfortable in engaging both technologists and users alike.

To grow and be successful in this role, you will ideally be/ bring the following:

• Proven experience in a senior or lead data engineering or data platform role.
• Deep expertise in designing, building, and operating scalable data services.
• Strong proficiency in Python and SQL is essential.
• Extensive experience with Apache Airflow, including orchestrating and managing complex DAGs, with a focus on memory-efficient patterns.
• Hands-on experience with distributed processing frameworks like Apache Spark.
• Experience with Kubernetes and data-related cloud native services.
• Solid understanding of data warehousing, data lakes, and data Lakehouse concepts.
• Knowledge of data governance, data security, and compliance principles.
• Excellent problem-solving skills and the ability to troubleshoot complex distributed systems.
• Strong communication and interpersonal skills, with the ability to lead and influence technical discussions.
• Experience mentoring junior to mid-level engineers.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• A pragmatic and can do attitude.
  
  

Desirable, but not a must:

• Experience with Openshift platforms
• Familiarity with monitoring and observability tools for data pipelines and infrastructure.

You’re not expected to have 100% of these skills. At ANZ a growth mindset is at the heart of our culture, so if you have most of these things in your toolbox, we’d love to hear from you.