Job Description

Key Responsibilities:
Design, build, and maintain robust ETL/ELT data pipelines using Apache
Spark on Databricks.
Implement data Lakehouse architecture using Delta Lake for cost-effective
data storage and analytics.
Use Databricks Workflows for orchestrating batch and streaming pipelines.
Develop and maintain CI/CD pipelines for data applications using tools such
as AWS, GitHub Actions or Databricks Repos.
Monitor pipeline performance and troubleshoot data issues in real-time and
batch environments.
Document solutions, workflows, and technical standards.

Required Skills:
Experience in data engineering with a strong focus on AWS Databricks and
Apache Spark.
Proficiency in PySpark, SQL, and Python.
Experience with Delta Lake, Databricks SQL, and Unity Catalog.
Hands-on experience with cloud platforms
Familiarity with data lakehouse architecture, data warehousing and
streaming data
Strong understanding of ETL best practices, data partitioning, and
performance tuning.
Experience with CI/CD for data pipelines.
Experience in Machine learning and Data Analytics is added advantage.
Strong exposure and understanding on the AI and Agentic AI technologies.
Excellent problem-solving and communication skills ##LI-DNI

Qualifications

Key Responsibilities:
Design, build, and maintain robust ETL/ELT data pipelines using Apache
Spark on Databricks.
Implement data Lakehouse architecture using Delta Lake for cost-effective
data storage and analytics.
Use Databricks Workflows for orchestrating batch and streaming pipelines.
Develop and maintain CI/CD pipelines for data applications using tools such
as AWS, GitHub Actions or Databricks Repos.
Monitor pipeline performance and troubleshoot data issues in real-time and
batch environments.
Document solutions, workflows, and technical standards.

Required Skills:
Experience in data engineering with a strong focus on AWS Databricks and
Apache Spark.
Proficiency in PySpark, SQL, and Python.
Experience with Delta Lake, Databricks SQL, and Unity Catalog.
Hands-on experience with cloud platforms
Familiarity with data lakehouse architecture, data warehousing and
streaming data
Strong understanding of ETL best practices, data partitioning, and
performance tuning.
Experience with CI/CD for data pipelines.
Experience in Machine learning and Data Analytics is added advantage.
Strong exposure and understanding on the AI and Agentic AI technologies.
Excellent problem-solving and communication skills