Job Description

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

ABOUT THE ROLE
WHAT YOU WILL DO
- Build and maintain scalable, distributed, fault-tolerant data pipelines on GCP;
- Develop and manage lakehouse layers and Delta Lake workflows using BigQuery and Dataproc;
- Collaborate with stakeholders across data engineering, compliance, and business teams;
- Design and implement pipelines to acquire, normalise, transform, and release large volumes of financial data;
- Design and implement bitemporal data models on BigQuery for regulatory-grade time-series datasets;
- Build and maintain testing frameworks for data pipelines and transformation logic;
- Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails;
- Contribute to shared platform services in a collaborative environment;
- Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search using Vertex AI.

MUST HAVES
-
6–8 years of experience in data engineering
;
-
Proficiency in Python
for data pipelines, transformation logic, and automation;
-
Proficiency in SQL
with hands-on experience in
BigQuery
including partitioning, clustering, and time-series queries;
-
Experience with Cloud Composer (Apache Airflow)
for pipeline orchestration;
-
Working knowledge of Dataproc (Apache Spark)
for batch ingestion and incremental processing;
-
Experience with AI-assisted development tools
such as GitHub Copilot or similar;
-
Experience with Git
version control and collaboration workflows;
- Familiarity with REST APIs for integrations;
-
Familiarity with GCP technologies
(Cloud Storage, Pub/Sub, Datastream, Cloud Monitoring, IAM, VPC Service Controls);
- Understanding of financial data concepts related to equities and& other asset classes;
-
Upper-intermediate English level.
NICE TO HAVES
- Knowledge of data libraries such as pandas or PySpark;
- Experience with columnar storage and time-series analytics tools such as ClickHouse;
- Familiarity with Dataplex for data governance and lineage;
- Understanding of Change Data Capture (CDC) using Datastream;
- Understanding of bitemporal data modeling concepts;
- Knowledge of financial reference data such as equities, fixed income, or corporate actions;
- Experience with BigQuery cost management techniques;
- Experience with CI/CD pipelines and Terraform for infrastructure as code;
- Exposure to LLMs and& Agentic AI using Vertex AI for data-related use cases.

PERKS AND BENEFITS
-
Remote work & Local connection:
Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts.
-
Legal presence in India:
We ensure full local compliance with a structured, secure work environment tailored to Indian regulations.
-
Competitive Compensation in INR:
Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness.
-
Innovative Projects:
Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.

About AgileEngine

AgileEngine is a digital consultancy offering top-1% tech talent across 15+ countries. We’ve helped 300+ clients in 17+ industries — from VC-backed startups to Fortune 500 brands — build software that solves real-world problems and drives measurable impact. Our clients achieve up to 3x higher ROI and 3x faster time-to-hire, with some partnerships spanning 10+ years.

Work with AgileEngine your way. Our flexible engagement models include staff augmentation and end-to-end product delivery, depending on your needs — ensuring faster scaling, seamless collaboration, and AI-enhanced productivity.

What we offer:

• Custom software development – scalable web, mobile, and backend solutions.

• AI & data solutions – GenAI, ML, and analytics for faster, smarter decisions.

• Design solutions – UX/UI consultancy and product design that wows users.

• Quality solutions – QA automation and testing for flawless delivery.

Our impact speaks for itself. AgileEngine has been listed on the Inc. 5000 for nine consecutive years, named among the Top 100 Software Development Companies of 2025 by Techreviewer, and recognized by Clutch as a Top Machine Learning Company.

Ready to build something exceptional? Schedule a free discovery call with our team!

Industry

IT & Software

Company Size

501-1,000 employees

Headquarters

Boca Raton, Florida

Year Founded

2010

Website

agileengine.com

Social Media

Data Engineer (GCP) ID56375

Job Description

About AgileEngine