Job Description

We are looking for a highly skilled Data Engineer to solve complex data challenges, optimize production data pipelines, and enhance database performance. You will play a critical role in diagnosing and debugging data related issues, ensuring seamless data flow, and enabling data-driven decision-making.

Key Responsibilities

• Diagnose and resolve data issues, production outages, and performance bottlenecks to maintain system reliability.
• Debug data job failures, including pipeline breakdowns and unexpected row data changes.
• Optimize database queries and monitor database health to improve efficiency and scalability.
• Develop workflows for automated data regeneration to maintain accuracy and consistency.
• Facilitate secure and efficient data access for analysis, balancing liberalization with compliance.
• Stay ahead of industry trends and recommend best-in-class data engineering technologies and practices.
• Work with the Data Lead and engineering team to implement optimal data ingestion and storage strategies.
• Design and build scalable, robust, and maintainable data pipelines in consultation with Data Lead using cutting-edge technologies.

Requirements

• A minimum of 4 years relevant working experience is preferred.
• Strong proficiency in Python with extensive hands on experience using Pandas for data manipulation, transformation, and analysis of large datasets.
• Strong expertise in SQL performance tuning, database optimization, and query efficiency
• Hands-on experience in developing, deploying, and debugging robust data pipelines and ETL/ELT workflows in a production environment.
• Experience with at least one cloud platform (AWS, GCP, Azure) and data warehousing solutions.
• Strong problem-solving skills, especially in navigating ambiguous and unknown data issues.
• Excellent communication skills, with the ability to collaborate effectively with engineering and product teams.