Job Description

Details

Business Title

Data Engineer

Years of Experience

Min 3 and max upto 7.

Job Descreption

Looking for a hands‑on Senior Data Engineer – Azure & Databricks with up to 6 years of total experience to build, optimize, and maintain scalable cloud data platforms.
This is an individual contributor role, focused on developing reliable data pipelines and analytics‑ready datasets using Microsoft Azure and Databricks. The role requires strong hands‑on expertise in SQL and Python, along with experience building batch (and basic streaming) data pipelines in a cloud environment.
You will collaborate closely with architects, analytics teams, QA, DevOps, and business stakeholders in a global delivery model.

Must have skillsHands‑on experience delivering Azure‑based data engineering solutions
Cloud & Data Engineering (Azure)

Strong hands‑on experience with Azure data services, including:
Azure Data Lake Storage Gen2 (ADLS)
Azure Data Factory (ADF)
Azure Databricks
Azure Synapse Analytics

Experience designing cloud‑native data lakes and analytical data stores
Solid understanding of batch data pipelines and basic streaming concepts

SQL & Python (Mandatory)
Strong SQL skills (mandatory)
Writing complex queries, joins, aggregations, and transformations
Hands‑on experience working with large datasets in Synapse / Databricks

Strong Python skills (mandatory)
Python for ETL / ELT and data engineering use cases
Hands‑on experience with PySpark in Databricks

Strong understanding of data modeling, transformations, and query performance tuning

Data Processing & Engineering
Hands‑on experience with Spark / PySpark on Databricks
Experience handling structured and semi‑structured data
Understanding of partitioning, schema evolution, and data validation concepts

DevOps & Platform Basics
Working knowledge of Infrastructure as Code (ARM Templates and/or Terraform)
Basic experience with CI/CD pipelines (Azure DevOps or GitHub Actions)
Understanding of logging and monitoring using Azure Monitor / Log Analytics

Collaboration
Ability to work effectively with architects, QA, DevOps, and business stakeholders
Good communication skills for explaining technical concepts clearly

Good to have skillsExposure to streaming technologies such as Azure Event Hubs or Kafka
Familiarity with Lakehouse architecture and Delta Lake concepts
Experience integrating Azure data platforms with BI tools (Power BI preferred)
Basic knowledge of data governance and data quality frameworks
Awareness of Azure cost‑optimization best practices
Experience working in Agile delivery models, especially with globalclients

Key responsibiltesData Engineering & Development

Build scalable ETL / ELT pipelines using Azure and Databricks
Develop SQL‑based transformations and Python / PySpark pipelines
Ingest and process data using ADF, Databricks, and ADLS
Build analytics‑ready data models optimized for performance and cost

Platform & Operations
Support deployment and execution of data pipelines across environments
Monitor data pipeline health, performance, and data quality
Troubleshoot pipeline failures and perform root‑cause analysis
Follow Azure security, reliability, and scalability best practices

Collaboration & Delivery
Work closely with architects and product teams to understand requirements
Translate business and analytics needs into working Azure data solutions
Contribute to documentation, code reviews, and engineering standards

Education Qulification

1. Bachelor’s or Master Degree or equivalent Degree

Certification If Any

1. Microsoft Certified: Fabric Data Engineer Associate
2.Microsoft Certified: Azure Solutions Architect Expert
3. Snowflake Core

Shift timing

12 PM to 9 PM and / or2 PM to 11 PM - IST time zone

Location:

DGS India - Pune - Kharadi EON Free Zone

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent