Job Description

We are looking for a skilled Data Engineer with around 2-4 years of experience to join our data engineering team. The ideal candidate should have strong hands-on experience in Snowflake, SQL, Python, PySpark, Spark Architecture and along with a solid understanding of data modeling and ETL pipeline development.

The candidate will be responsible for designing, developing, and maintaining scalable data pipelines, transforming and processing large datasets, and implementing data solutions using Snowflake and distributed data processing technologies. This role requires strong problem-solving skills and the ability to work effectively with cross-functional teams.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines for processing and integrating data from multiple sources.
• Develop efficient and optimized SQL queries for data extraction, transformation, and analysis.
• Build and maintain data processing solutions using Python, PySpark/Snowpark.
• Develop and manage data solutions on Snowflake, including data loading, transformation, and data processing.
• Design and implement effective data models to support analytical and reporting requirements.
• Perform data transformation, cleansing, validation, and quality checks to ensure data accuracy and consistency.
• Optimize SQL queries, Spark jobs, and data pipelines for performance and scalability. Troubleshoot and resolve issues related to data pipelines, transformations, and data processing.
• Collaborate with business analysts, developers, and other stakeholders to understand data requirements and deliver reliable data solutions.
• Follow engineering best practices for code quality, version control, testing, documentation, and deployment.

Qualifications

• Bachelor’s degree in computer science, Information Technology, Engineering, or a related field.
• Around 2-4 years of professional experience in Data Engineering or a related field.
• Mandatory: Strong hands-on experience with Snowflake and Snowpark including its data warehousing capabilities.
• Mandatory: Strong hands-on experience with internal and external stages, file formats, data loading, Snowpipe, Streams, Tasks, stored procedures, and Snowflake security/access controls.
• Mandatory: Experience with Snowpark, particularly using Python/Snowpark for data processing and transformation within Snowflake.
• Mandatory: Good understanding of Snowflake architecture, including virtual warehouses, databases, schemas, compute/storage separation, and basic performance and cost optimization techniques.
• Mandatory: Familiarity with cloud storage technologies such as Amazon S3 or Azure Data Lake Storage.
• Mandatory: Strong hands-on experience with SQL.
• Mandatory: Strong programming experience in Python and hands-on experience with PySpark for large-scale data processing.
• God understanding of data modeling concepts, including relational and dimensional modeling.
• Experience in designing and developing ETL/ELT pipelines.
• Strong understanding of data transformation, data integration, and data processing concepts.
• Good analytical and problem-solving skills.
• Ability to work independently as well as collaboratively in a team environment.

Nice to have:

• Experience with Apache Airflow or other workflow orchestration tools.
• Knowledge of dbt and modern ELT practices.
• Experience with Git and CI/CD practices.
• Familiarity with data lake and cloud storage technologies such as Amazon S3 or Azure Data Lake Storage.
• Understanding of Snowflake performance optimization, warehouse sizing, clustering, and query optimization.
• Experience working with large-volume datasets and distributed data processing.
• Snowflake SnowPro Core or Advanced certification.
• Experience with CI/CD pipelines and DevOps practices.
• Knowledge of data cataloging tools and metadata management.
• Exposure to BI tools like Power BI, Tableau or Looker.
• Excellent communication and stakeholder management skills.