About the Role:
An offshoring company is looking for a Data Engineer.
Responsibilities:
Design, build, and manage the complete orchestration framework for all data pipelines using Google Cloud Composer.
Author complex, dynamic, and maintainable DAGs in Python, implementing sophisticated dependency management, triggers, and scheduling logic.
Serve as the team's subject matter expert on all DAG Authoring/Best Practices, including idempotency, task modularity, backfilling, and performance tuning.
Expert-Level DAG Authoring: Demonstrable mastery of authoring complex DAGs in Python. This must include deep knowledge of Airflow operators and sensors, dynamic DAG generation, managing task dependencies and SLAs, and creating idempotent tasks.
Cloud Composer (Apache Airflow): Deep, hands-on experience managing a production Cloud Composer environment, including configuration, security, scaling, and troubleshooting.
Python for Orchestration: Expert-level Python proficiency as it applies to writing clean, efficient, and testable Airflow DAGs.
Requirements:
5+ years strong hands-on experience with Python and Cloud Composer (Apache Airflow): configuration, security, scaling, troubleshooting.
Expert in Python for writing clean, efficient, testable Airflow DAGs.
Experience with CI/CD for data pipelines (Git, Google Cloud Build).
Shift: Night shift
Setup: Remote
By Applying, you give consent to collect, store, and/or process personal and/or sensitive information for the purpose of recruitment and employment may it be internal to Cobden & Carter International and/or to its clients.
