Blue Pearl HQ

Azure Data Engineer

Blue Pearl HQ  •  Johannesburg, ZA (Hybrid)  •  3 months ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description


The Azure Data Engineer will be responsible for designing, building, optimizing, and maintaining data pipelines and data solutions within the Microsoft Azure ecosystem. The role requires strong experience in cloud-based data engineering, modern data architectures, and scalable data processing frameworks.


Key Responsibilities


  • Design, develop, and maintain
    Azure-based data pipelines

    (batch and streaming).

  • Implement ETL/ELT solutions using
    Azure Data Factory (ADF), Azure Databricks, Synapse Analytics

    , and related Azure services.

  • Build and optimize
    data models

    , including Data Lakehouse architectures (Medallion, Delta Lake).

  • Ensure data quality, integrity, and governance across all data processes.

  • Collaborate with data scientists, analysts, and business teams to provide clean, reliable datasets.

  • Perform pipeline monitoring, troubleshooting, performance optimization, and cost optimization.

  • Develop CI/CD pipelines using
    Azure DevOps

    for data-related deployments.

  • Implement security best practices, including
    RBAC, encryption, and data access governance

    .


Required Technical Skills


  • Strong proficiency with
    Azure Data Factory (ADF)

    ,
    Azure Databricks (PySpark)

    , and
    Azure Synapse Analytics

    .

  • Expert knowledge of
    Python

    ,
    SQL

    , and data transformation frameworks.

  • Experience with
    Delta Lake

    ,
    Spark

    ,
    Data Lake Storage (ADLS)

    , and distributed compute patterns.

  • Familiarity with
    Data Vault

    ,
    Dimensional Modelling

    , or
    Medallion architecture

    .

  • Experience with cloud DevOps tools (
    Azure DevOps, GitHub Actions

    ).

  • Knowledge of
    streaming technologies

    (Kafka, Event Hubs) is an advantage.


Requirements


Qualifications & Certifications (Preferred)


  • Azure certifications such as:

    • AZ‑900 Azure Fundamentals

    • DP‑203 Azure Data Engineer Associate

    • Databricks Data Engineer certifications

  • Bachelor’s degree in Computer Science, Engineering, or related field.


Experience


  • 5+ years of data engineering experience (Azure preferred).

  • Proven experience building and supporting production-scale cloud data solutions.


Location & Working Model


  • Location:

    Johannesburg (Hybrid working model)

  • Ways of Working:

    Standard SAST business hours

  • Candidate:

    Preferably a
    South African Citizen or Permanent Resident
Blue Pearl HQ

About Blue Pearl HQ

Blue Pearl is a market-leading CLOUD Solutions developer with extensive knowledge and insight into the latest technologies, standardised processes, advanced technical capabilities and consulting processes available, ensuring wholistic success for our clientele. We offer professional consulting to compliment your business strategy and overall management and make it our priority to add value to any business by listening, analysing and creating a conducive solution that will empower our client.

We implement a Data Analysis Process that includes inspecting, cleansing, transforming, and modelling data with the end-goal of discovering useful information, informing conclusions, and relevant information to support your decision-making. Your business cannot afford not to engage with us, allowing our data analysis to play a role in making your business decisions more scientific and helping your business achieve effective operation.

Blue Pearl’s team of experts include BI strategists, BI analysts, Data Warehouse Architects, Data Scientists, Implementation and Development experts. With the use of BI, Analytics and Big Data, we effectively partner with our customers on their mission to achieve a competitive business advantage and real ROI from the structured information we collect.

Industry
IT & Software
Company Size
11-50 employees
Headquarters
Johannesburg, ZA
Year Founded
2013
Social Media