Role Overview

We are looking for a highly skilled Data Quality Engineer with strong Data Engineering expertise to ensure the accuracy, reliability, and scalability of enterprise data platforms. The ideal candidate will possess hands-on experience with Databricks, PySpark, Hadoop, Hive, and Cloud technologies, along with advanced SQL skills to validate data across large-scale data pipelines and Lakehouse architectures.

Required Experience

• 
  3+ years of experience in Data Engineering, Data Quality Engineering, or Data Testing.
• 
  Hands-on experience with Databricks and PySpark.
• 
  Strong experience in Hadoop ecosystem components such as Hive, HDFS, Spark, and related Apache technologies.
• 
  Advanced SQL expertise for large-scale data validation and analysis.
• 
  Experience working with Data Warehouses, Data Lakes, and Lakehouse architectures.
• 
  Understanding of Star Schema, Snowflake Schema, and dimensional modeling.
• 
  Experience with cloud platforms (Azure, AWS, or GCP).

Key Responsibilities

Data Quality & Validation

• 
  Perform end-to-end validation of data pipelines across ingestion, transformation, and consumption layers.
• 
  Execute source-to-target reconciliation and data quality checks.
• 
  Identify, investigate, and resolve data anomalies and inconsistencies.
• 
  Define and implement data quality frameworks, metrics, and controls.

Data Engineering & Processing

• 
  Develop and validate data pipelines using PySpark and Databricks.
• 
  Work with large-scale datasets in Hadoop, Hive, and Lakehouse environments.
• 
  Support ETL/ELT workflows and ensure data integrity throughout the data lifecycle.
• 
  Optimize data processing jobs for performance and scalability.

SQL & Analytics

• 
  Write advanced SQL queries for data profiling, reconciliation, and root cause analysis.
• 
  Perform complex joins, window functions, CTEs, aggregations, and query optimization.
• 
  Validate business rules and transformation logic against source systems.

Lakehouse & Cloud Platforms

• 
  Validate and monitor data across Databricks Lakehouse architecture.
• 
  Work with cloud platforms such as Azure, AWS, or GCP.
• 
  Collaborate with Data Engineers, Architects, and Analysts to ensure reliable data delivery.

Defect & Incident Management

• 
  Analyze production issues and conduct root cause analysis.
• 
  Track and manage data defects through resolution.
• 
  Implement proactive monitoring and automated quality checks.

Required Experience

• 
  6+ years of experience in Data Engineering, Data Quality Engineering, or Data Testing.
• 
  Hands-on experience with Databricks and PySpark.
• 
  Strong experience in Hadoop ecosystem components such as Hive, HDFS, Spark, and related Apache technologies.
• 
  Advanced SQL expertise for large-scale data validation and analysis.
• 
  Experience working with Data Warehouses, Data Lakes, and Lakehouse architectures.
• 
  Understanding of Star Schema, Snowflake Schema, and dimensional modeling.
• 
  Experience with cloud platforms (Azure, AWS, or GCP).

Preferred Skills

• 
  Automated data testing frameworks.
• 
  Data observability and monitoring tools.
• 
  CI/CD implementation for data pipelines.
• 
  Experience with Delta Lake, Unity Catalog, or similar technologies.
• 
  Knowledge of Airflow, Kafka, or other Apache ecosystem tools.