We are looking for an experiencedAWS Data Engineer with 4–8 years of hands-on experience in designing, developing, and maintaining scalable data pipelines and cloud-based data platforms.

The ideal candidate will have strong expertise inAWS, Snowflake, Apache Airflow, Python,PySpark, and SQL , with a solid understanding of data warehousing, ETL/ELT, data modeling, and performance optimization.

The candidate will work closely with data architects, analysts, application teams, and business stakeholders to build reliable and scalable data solutions.

Responsibilities

Design, develop, and maintain scalabledata pipelines and ETL/ELT workflows using AWS services.

Build and orchestrate data pipelines usingApache Airflow , including DAG development, scheduling, monitoring, retries, dependencies, and error handling.

Develop data processing and transformation solutions usingPython andPySpark

Design and implement data warehouse solutions usingSnowflake

Develop complexSQL queries, stored procedures, views, CTEs, and data transformations

Work with AWS services such asS3, Glue, Lambda, EMR, Athena, Redshift, and IAM

Build batch and, where required, near-real-time data ingestion pipelines.

Implement data ingestion from APIs, databases, files, and other source systems into AWS/Snowflake.

PerformSnowflake performance and cost optimization , including warehouse sizing, query optimization, clustering, partitioning, and efficient data loading.

Implement Snowflake features such asSnowpipe, Streams, Tasks, stages, file formats, and secure data sharing

Develop scalable Spark/PySparkjobs and optimize transformations,joins, partitioning, caching, and resource utilization.

Implement data quality checks, validation, reconciliation, and monitoring mechanisms.

Troubleshootpipeline failures, data issues, performance bottlenecks, and production incidents.

Follow best practices fordata security, governance, access control, and PII-sensitive data handling

UseGit and CI/CD practices for source control, automated testing, and deployment of data pipelines.

Collaborate with cross-functional teams in an Agile/Scrum environment.

Create technical documentation for data pipelines, workflows, data models, and operational procedures.

Mandatory Skill sets:

4–8 years of experience in Data Engineering.

Strong hands-on experience withAWS Data Engineering

Strong experience withSnowflake

Hands-on experience withApache Airflow and DAG development.

Strong programming experience inPython

Strong hands-on experience withPySpark Apache Spark

AdvancedSQL skills.

Strong understanding ofETL/ELT and data pipeline development

Experience working withAWS S3 and AWS Glue

Good understanding ofdata warehousing and dimensional data modeling

Experience with data pipeline monitoring, debugging, and performance optimization.

Good understanding ofGit and CI/CD

Preferred Skill sets:

AWSLambda, EMR, Athena, Redshift, Kinesis, Step Functions, IAM

SnowflakeSnowpipe, Streams, Tasks, Dynamic Tables, Time Travel and performance tuning

Experience withdbt

Experience withKafka or other streaming technologies.

Experience withTerraform / Infrastructure as Code

Experience with data quality tools such asGreat Expectations

Knowledge ofLakehouse / Medallion Architecture

Experience withDatabricks

Snowflake certification such asSnowProCore

Exposure to Docker/Kubernetes is a plus.