AI Success™

Job Description

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

ABOUT THE ROLE
As a Data Engineer specializing in Java and Apache Spark, you will help build and evolve large-scale data processing systems that power experimentation and user insights. Working within a cloud-based AWS EMR environment, you’ll contribute to improving data infrastructure reliability, scalability, and cost efficiency. This role offers the opportunity to shape critical datasets and analytics capabilities while collaborating with platform and data teams to support high-impact experimentation and decision-making.

WHAT YOU WILL DO
- Develop and maintain index builder products, including user session index builders, user session–derived index builders, and experimentation platform index builders;
- Investigate and resolve reported issues related to index builders;
- Assist with user inquiries regarding the platform and its datasets;
- Improve index builder stability and reliability;
- Support efforts to optimize compute costs across the platform;
- Contribute to the Central Exposure Dataset effort, including building a consolidated dataset for experiment analysis;
- Work toward meeting code freshness goals;
- Persist Yarn logs and Spark history for terminated clusters;
- Capture metrics from UserCohort;
- Optimize resource allocation for platform infrastructure;
- Help reduce the number of core instances for platform clusters;
- Support the deprecation of legacy index builders used for experiment analysis.

MUST HAVES
-
4+ years experience in software development
;
-
Bachelor’s degree in Computer Science or equivalent practical experience
;
-
Significant practical experience with Java (4+ years)
;
-
Practical experience implementing Apache Spark jobs
, including partitioning, grouping, joins, importing data into the cluster, and exporting data from the cluster;
-
Practical experience working with AWS, specifically AWS EMR
(or ability to pick it up fast);
-
Upper-intermediate English level
.

NICE TO HAVES
- Basic knowledge of Kubernetes;
- Experience with Spark Operator;
- Experience with Airflow;
- Experience with Scala.

PERKS AND BENEFITS
-
Professional growth:
Mentorship, TechTalks, and personalized growth roadmaps.
-
Competitive compensation:
USD-based pay with education, fitness, and team activity budgets.
-
Exciting projects:
Modern solutions with Fortune 500 and top product companies.
-
Flextime:
Flexible schedule with remote and office options.

About AgileEngine

AgileEngine is a digital consultancy offering top-1% tech talent across 15+ countries. We’ve helped 300+ clients in 17+ industries — from VC-backed startups to Fortune 500 brands — build software that solves real-world problems and drives measurable impact. Our clients achieve up to 3x higher ROI and 3x faster time-to-hire, with some partnerships spanning 10+ years.

Work with AgileEngine your way. Our flexible engagement models include staff augmentation and end-to-end product delivery, depending on your needs — ensuring faster scaling, seamless collaboration, and AI-enhanced productivity.

What we offer:

• Custom software development – scalable web, mobile, and backend solutions.

• AI & data solutions – GenAI, ML, and analytics for faster, smarter decisions.

• Design solutions – UX/UI consultancy and product design that wows users.

• Quality solutions – QA automation and testing for flawless delivery.

Our impact speaks for itself. AgileEngine has been listed on the Inc. 5000 for nine consecutive years, named among the Top 100 Software Development Companies of 2025 by Techreviewer, and recognized by Clutch as a Top Machine Learning Company.

Ready to build something exceptional? Schedule a free discovery call with our team!

Industry

IT & Software

Company Size

501-1,000 employees

Headquarters

Boca Raton, Florida

Year Founded

2010

Website

agileengine.com

Social Media

Data Engineer (Senior) ID53687

Job Description

About AgileEngine