Job Description

We are looking for a Senior Databricks Engineer with 7+ years of experience in data engineering and strong hands-on expertise in Databricks and Apache Spark . The ideal candidate will design and build scalable data pipelines, optimize large-scale data processing, and work with cloud-based data platforms. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark . Develop and optimize PySpark/Spark jobs for large-scale data processing. Build reliable ETL/ELT pipelines using Databricks workflows, Delta Lake, and SQL . Implement data transformations, data quality checks, and performance optimization. Work with cloud data platforms such as AWS, Azure, or GCP . Optimize Spark jobs, cluster configurations, partitioning, joins, and data processing performance. Collaborate with data engineers, data scientists, analysts, and business teams to deliver data solutions. Implement best practices for CI/CD, version control, monitoring, and production support . Troubleshoot data pipeline and performance issues in production environments. Required Skills 7+ years of experience in Data Engineering . Strong hands-on experience with Databricks . Strong expertise in Apache Spark / PySpark . Experience with Delta Lake, Databricks Workflows, and Spark SQL . Strong proficiency in Python and SQL . Experience building and optimizing large-scale ETL/ELT pipelines. Experience with at least one cloud platform: AWS, Azure, or GCP . Good understanding of data warehousing, data modeling, and distributed data processing. Experience with Git and CI/CD practices. Preferred Skills Experience with Unity Catalog and Databricks governance. Experience with Azure Data Factory, AWS Glue, or similar orchestration tools . Knowledge of streaming technologies such as Kafka or Spark Structured Streaming . Experience with Infrastructure as Code or cloud automation. Databricks certifications are a plus.