Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ey Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ________________________________________ Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ________________________________________ Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues.Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Job Title: Senior Data Engineer (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
HCLTech is a global technology company, home to more than 226,600 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending September 2025 totaled $14.2 billion. To learn how we can supercharge progress for you, visit hcltech.com.
