AWS Data Engineer- Airflow

Years of Experience: 5-7 Years

We are seeking a highly motivated and detail-oriented Data Consultant who is flexible, proactive, and capable of working independently. The ideal candidate will possess strong communication and coordination skills, with a high level of diligence and attention to detail. This role requires collaboration with cross-functional teams to ensure seamless project execution while demonstrating strong ownership and the ability to work with minimal supervision.

Responsibilities:

• Collaborate effectively with cross-functional teams to ensure seamless project execution.
• Proactively identify issues and implement timely solutions.
• Maintain high standards of accuracy and attention to detail in all deliverables.
• Utilize SQL and data warehousing concepts to manage and analyze data.
• Work with Amazon Redshift and other AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
• Develop and manage workflows using Airflow (DAGs).
• Implement MLOps practices and utilize PySpark and Python for data processing.
• Utilize version control tools such as GitHub, including Copilot, for project management.

Mandatory Skills:

• Strong experience in SQL and data warehousing concepts.
• Hands-on experience with Amazon Redshift.
• Expertise in workflow orchestration tools such as Airflow (DAGs).
• Experience with AWS services including Glue Jobs, Step Functions, RDS, and SageMaker.
• Proficiency in PySpark and Python.
• Understanding of MLOps practices.
• Familiarity with version control tools such as GitHub (including Copilot).

Preferred Skills:

• Experience with SSIS and Netezza.
• Knowledge of BMO processes.
• Familiarity with AWS CDK (Cloud Development Kit).

Qualifications:

• Bachelor's degree in computer science, Information Technology, or a related field.
• 5-7 years of relevant experience in data consulting or a similar role.