Job Description

As a Principal Data Engineer, you will design, build, and maintain scalable data pipelines and infrastructure to support analytics, reporting, and data science initiatives. You will work closely with cross-functional teams to ensure data is accessible, reliable, and secure across the organization.

Primary Job Duties and Responsibilities (Essential Job Functions)

1. Design and Develop Scalable Data Pipelines
   • Design and implement scalable data ingestion and transformation frameworks using Azure services enabling structured, semi-structured, and unstructured data to be efficiently processed and integrated into enterprise data platforms
   • Build and maintain robust ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
   • Integrate data from diverse sources including on-premises systems, cloud storage, APIs, and streaming platforms.
2. Databricks Development and Optimization
   • Develop and optimize notebooks and workflows in Azure Databricks using PySpark, SQL.
   • Implement Delta Lake for efficient data storage, versioning, and ACID transactions.
   • Leverage Databricks features such as Unity Catalog and job orchestration.
3. Data Modeling and Architecture
   • Design and implement data models (star/snowflake schemas) for analytics and reporting.
   • Collaborate with architects to define data lakehouse architecture and best practices.
   • Hands-on experience implementing and optimizing data solutions using the Medallion Architecture (Bronze, Silver, Gold layers) for scalable and structured data processing
4. Data Quality and Governance
   • Implement data validation, profiling, and cleansing routines.
   • Ensure compliance with data governance policies, including data lineage and metadata management.
5. Performance Tuning and Monitoring
   • Monitor and optimize performance of Spark jobs and data pipelines.
   • Troubleshoot and resolve issues related to data latency, job failures, and resource utilization.
6. Collaboration and Stakeholder Engagement
   • Work closely with data scientists, analysts, and business units to understand data requirements.
   • Translate business needs into technical solutions that are scalable and maintainable.
7. Security and Compliance
   • Implement role-based access control (RBAC), encryption, and secure data handling practices.
   • Ensure compliance with industry regulations (e.g., NERC CIP, GDPR, HIPAA if applicable).
8. Documentation and Best Practices
   • Maintain clear documentation of data flows, architecture, and operational procedures.
   • Promote best practices in code versioning, testing, and CI/CD for data engineering.

• Bachelor’s degree in information systems, Computer Science, or a related technical field; or equivalent work experience.
• 10 years of experience with advanced knowledge of data architecture, cloud platforms (especially Azure), and enterprise data solutions.
• Advanced understanding of data modeling, ETL/ELT processes, and performance tuning of enterprise-level applications.
• Expert-level knowledge of data-related technologies from architecture to administration, including design, development, optimization, and licensing.
• Proven experience working in the utility industry is required
• Soft Skills:
• Effective oral and written communication skills, with the ability to collaborate across teams and mentor junior engineers.
• Strong analytical and problem-solving abilities.
• Ability to prioritize and manage multiple tasks and projects concurrently.

MidAmerican Energy Company, a Midwest utility, provides regulated electric and natural gas service to more than 1.6 million customers in Illinois, Iowa, Nebraska and South Dakota. The company owns and operates a portfolio of power-generating assets, approximately 61% of which is wind generation.