Senior Data Engineer
JD Design develop and maintain scalable ETL pipelines using AWS Glue PySpark and AWS Lambda
Migrate structured and semistructured data from existing data lakes eg S3 Hadoop to Amazon Redshift
Optimize data ingestion and transformation processes for performance and costefficiency
Collaborate with data architects analysts and business stakeholders to understand data requirements and ensure data quality
Implement data validation cleansing and transformation logic
Monitor and troubleshoot data pipelines and workflows
Ensure data security and compliance with organizational and regulatory standards
Document data flows architecture and processes Responsibility
Design develop and maintain scalable ETL pipelines using AWS Glue PySpark and AWS Lambda
Migrate structured and semistructured data from existing data lakes eg S3 Hadoop to Amazon Redshift
Optimize data ingestion and transformation processes for performance and costefficiency
Primary Skills
AWS Glue jobs
AWS services Lambda S3
AWS database Hadoop and Redshift
Communication
Usage of AI code assist tools
Working knowledge of implementing data migration
Virtusa is a global product and platform engineering services company that makes experiences better with technology. We help organizations grow faster, more profitably, and more sustainably by reimagining enterprises through domain-driven solutions. We combine strategy, design, and engineering, backed by unmatched expertise at the intersection of industry, business, and technology to generate real-world business impact for clients.
Headquartered in Massachusetts with global delivery centers, Virtusa provides a broad range of services, solutions, and assets, including strategy and design, AI advisory and services, digital engineering, data and analytics, digital assurance, cloud and security, cx transformation and managed services across industries such as financial services, healthcare, communications, media, entertainment, travel, manufacturing, and technology.
