Job Description

Key Responsibilities

Design and Build Data Pipelines Develop and maintain scalable data pipelines that automate the extraction transformation and loading ETL of data from various sources into centralized repositories

Data Management Create and manage data management systems ensuring they meet business requirements and industry practices

Data Quality Assurance Implement validation rules and cleansing procedures to maintain high data quality and integrity ensuring accurate and reliable data for analysis

Collaboration Work closely with data architects data scientists and analysts to understand their data needs and tailor solutions accordingly

Research and Development Explore opportunities for data acquisition and new uses for existing data integrating new technologies and tools into existing structures

Documentation and Reporting Document data processes and workflows and report on data quality and pipeline performance to stakeholders

Required Skills and Qualifications

Technical Proficiency Strong programming skills in languages such as Python Java or Scala and experience with SQL and NoSQL databases

Big Data Technologies Familiarity with big data tools and frameworks like Hadoop Spark and cloud platforms eg AWS GCP Azure

Data Architecture Understanding of data architecture principles and experience in building data warehouses and data lakes

Problemsolving Skills Ability to troubleshoot data pipeline issues and optimize performance

Communication Skills Strong verbal and written communication skills to collaborate with crossfunctional teams and present findings