Job Description

Job Summary

We are looking for a Data Lead – Data Engineering to drive technical diagnostics, optimization, and stabilization of enterprise data platforms supporting large-scale ETL and batch processing systems

Key Responsibilities

1. To be responsible for providing technical guidance to a team of developers, enhancing their technical capabilities and increasing productivity.
2. To conduct comprehensive code reviews, establish and oversee quality assurance processes, performance optimization , implementation of best practices and coding standards to ensure successful delivery of complex projects.
3. To ensure process compliance in the assigned module| and participate in technical discussions/review as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).
4. To collaborate with stakeholders to define project scope, objectives, deliverables and accordingly prepare and submit status reports for minimizing exposure & closure of escalations.

Skill Requirements

Strong expertise in: Azure Data Factory (ADF),Databricks (PySpark / Spark SQL preferred),SSIS and SQL Server Hands-on experience with: Batch orchestration tools (Control-M / Autosys),ETL/ELT pipeline design and optimization Strong experience in: SQL development and performance tuning, Troubleshooting and debugging complex data pipelines Familiarity with: Monitoring and alerting tools (Azure Monitor, Log Analytics) Exposure to large-scale data processing and batch system Review and optimize ADF pipelines, SSIS packages, and Databricks workflows for performance, scalability, and Analyze batch scheduling and orchestration frameworks (Control-M / Autosys), including dependencies, triggers, and SLA adherence Investigate and resolve pipeline failures, job delays, and runtime issues, ensuring faster recovery and minimal business Evaluate source-to-target integration flows, SQL logic, and transformation layers for efficiency and correctness Define and improve error handling, retry mechanisms, restart ability, and reprocessing strategies across ETL and batch Optimize runtime performance of data pipelines, including query tuning and batch execution improvements Establish and enhance monitoring and alerting frameworks using tools like Azure Monitor, Control-M dashboards, and custom logging solutions Identify automati

Other Requirements

null