Job Description

We are recruiting for a Databricks Application Developer in the Legal, Risk Management and Compliance (LRMC) team.

Roles & responsibilities

We are recruiting for a Databricks Application Developer in the Legal, Risk Management and Compliance (LRMC) team.

• Develop business applications and AI-enabled solutions using Databricks, translating solution requirements into working applications and integrations.
• Build Databricks applications and develop application logic, APIs, services, and integrations with enterprise systems.
• Design and implement scalable batch and streaming data pipelines using Python, SQL, PySpark, and Delta Lake.
• Apply Medallion Architecture principles across Bronze, Silver, and Gold layers to improve data quality, reusability, and consumption.
• Create and manage Databricks Workflows, jobs, task dependencies, parameters, scheduling, monitoring, and failure handling.
• Integrate governed data assets from Unity Catalog and apply appropriate access controls, lineage, and data governance practices.
• Design database objects and data models, optimize SQL queries, and improve performance of data processing workloads.
• Integrate Databricks applications with governed data, APIs, analytics services, and AI or machine learning capabilities.
• Use GitHub for source control and implement CI/CD pipelines with GitHub Actions for automated testing and deployment.
• Support unit testing, system integration testing, deployment, production readiness, troubleshooting, and technical documentation.

Mandatory technical & functional skills

• 3-5 years of experience in data engineering, data platform development, or Databricks application development.
• Hands-on expertise with Azure Databricks, Python, PySpark, Spark SQL, SQL, Delta Lake, and notebook-based development.
• Experience creating reliable ETL/ELT pipelines for batch and streaming data processing, including validation and error handling.
• Strong understanding of Medallion Architecture, dimensional data modelling, database design, indexing, query optimization, and performance tuning.
• Experience building Databricks applications, backend logic, REST APIs, and integrations with enterprise systems and data services.
• Hands-on experience with Databricks Workflows, jobs, scheduling, task orchestration, monitoring, logging, and production support.
• Experience with Unity Catalog, access controls, data lineage, governance, secrets, and secure data handling.
• Proficiency with GitHub source control, branching, pull requests, code reviews, GitHub Actions, and CI/CD deployment pipelines.
• Understanding of automated testing, Agile delivery, technical documentation, and operational readiness.

Preferred technical & functional skills

• Experience with Databricks Asset Bundles, Databricks SQL, Lakehouse architecture, and Infrastructure as Code.
• Exposure to Azure services such as Azure Data Lake Storage, Key Vault, Data Factory, Entra ID, and Azure DevOps.
• Experience integrating machine learning, generative AI, or governed AI services into Databricks applications.
• Knowledge of Power BI or other analytics and data visualization platforms.
• Understanding of risk management, compliance, governance, or other regulated business domains.