Job Description

Role & Responsibilities

• Serve as the Lead Cloud Engineer for customer engagements.
• Collaborate closely with the Lead Data Consultant throughout project delivery.
• Define solution architecture and detailed designs for data platforms on Google Cloud Platform (GCP).
• Build and deliver solutions, including hands-on development and coding based on approved designs.
• Engage directly with the client's Data Engineering team, providing consulting, guidance, and technical leadership on both solution architecture and implementation.
• Perform pair programming with client Data Engineers and facilitate knowledge transfer of the developed solution to the client team.

Minimum Qualifications

Solution Architecture Knowledge & Expertise

• Experience designing data platform architectures for:
  • Migration of open-source or public cloud data platforms to the cloud, preferably Google Cloud Platform (GCP).
  • Greenfield cloud-native data platform implementations.
• Ability to research tools and technologies, identify key evaluation criteria, and provide technology selection recommendations.
• Experience defining solution architectures and detailed designs for:
  • Batch ingestion and processing from sources such as files, cloud databases, and on-premises databases.
  • Streaming solutions for real-time ingestion and processing, including log analytics, clickstream analytics, and other real-time data sources.
• Experience building data pipelines and CI/CD pipelines.
• Deep knowledge of at least one public cloud platform, preferably GCP.

Tools & Technologies

• Google Cloud Platform (preferred); equivalent experience in other public cloud platforms is acceptable.
• Hands-on experience with:
  • Dataflow
  • Pub/Sub
  • Cloud Composer
  • BigQuery
  • Cloud Run
  • Cloud Functions
  • Spanner
  • Cloud Build
  • Vertex AI
  • Terraform

Coding Expertise

• Strong programming skills and willingness to work hands-on as part of daily responsibilities.
• Proficiency in at least one of the following programming languages, preferably both:
  • Python
  • Java

Preferred Qualifications

Solution Architecture

• Experience with open-source ecosystems and distributions such as:
  • Hadoop
  • Spark
  • Cloudera
  • Hortonworks
• Understanding of and experience with NoSQL databases, including:
  • HBase
  • MongoDB

Tools & Technologies

• Experience with additional public cloud platforms.
• Experience with open-source frameworks and ecosystems, including:
  • Hadoop
  • Spark
  • Oozie
  • Kafka
  • HBase

Certifications

At least one of the following certifications:

• Google Cloud Professional Data Engineer (preferred).
• AWS Certification.