Job Description

Responsibilities:

• Design, develop, test, and maintain scalable big data pipelines on Databricks and AWS.
• Build and manage data lake and warehouse models using dbt and Spark, following medallion/lakehouse architecture patterns.
• Own pipeline reliability: monitoring, alerting, incident response, and cost optimisation.
• Collaborate with game developers, analysts, and product teams to translate data requirements into robust infrastructure.
• Support BI and analytics development with well-structured, documented data models.
• Manage AWS data infrastructure including IAM, S3, Glue, and relevant data services.
• Contribute to data governance practices: data quality checks, schema management, lineage, and access controls.
• Review peers’ code and contribute to team standards around testing, CI/CD, and deployment via Databricks Asset Bundles.
• Work with stakeholders across the organisation to ensure scalability and performance as data volumes grow.

Required Skills & Experience:

• 4-5 years of professional data engineering experience.
• Advanced SQL Complex transformations, performance tuning, analytical query design.
• Apache Spark Hands-on production experience with PySpark or Spark SQL; understanding of distributed execution, partitioning, and performance.
• Databricks: Building and operating pipelines on the Databricks platform (Delta Lake, workflows, Unity Catalog).
• Big data pipeline design: Batch and/or streaming at scale; schema evolution; idempotency; reliability patterns.
• AWS infrastructure Practical experience with S3, IAM, Glue, Lambda, or equivalent services; comfortable reading and modifying infrastructure config.
• Python: For pipeline logic, automation, and data tooling.
• Data engineering best practices CI/CD for data, testing strategies (unit + integration), version control (Git), documentation-as-code.
• Team collaboration: Clear communicator; able to work across engineering, analytics, and product; comfortable in async and written communication.
• Basic AI enthusiasm: An interest in how AI/ML fits into modern data platforms and a willingness to learn; you don’t need to be an ML engineer.

Nice to Haves:

• AWS Redshift or RDS: Experience with Redshift Serverless or RDS as part of a data stack.
• Databricks Asset Bundles: Hands-on with DABs for CI/CD deployment and environment promotion.
• Dbt: Model authoring, Testing, and Documentation.
• Orchestration tooling: Airflow, Databricks Workflows, or equivalent.
• Streaming Kafka, Kinesis, or Spark Structured Streaming in production.
• Game analytics or mobile data: Familiarity with high-volume event instrumentation pipelines is a bonus, not a requirement.

What We’re Looking For (Beyond the Tech):