Guardian Life

Engineer - IT

Guardian Life  •  Chennai, IN (Onsite)  •  3 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Databricks and PySpark
  • Build robust ETL/ELT workflows for batch and near real‑time data processing
  • Write optimized and efficient SQL queries for data transformation, validation, and analysis
  • Work with large-scale distributed datasets using Spark-based processing frameworks
  • Perform data ingestion from multiple sources (APIs, files, databases) into data lake/lakehouse architecture
  • Ensure data quality, integrity, and consistency through validations and monitoring
  • Optimize pipeline performance and cost (partitioning, caching, query tuning, etc.)
  • Collaborate with cross-functional teams (Data Ops, Business, Analytics) to deliver data solutions
  • Troubleshoot pipeline failures and implement proactive monitoring and alerting
  • Maintain proper documentation for data models, transformations, and workflows

Required Skills & Experience

  • Strong hands-on experience in:
    • Databricks (Lakehouse platform, notebooks, jobs, clusters)
    • PySpark / Apache Spark
    • Advanced SQL (joins, window functions, performance tuning)
  • Solid understanding of ETL concepts, data warehousing, and data modeling
  • Experience building scalable data pipelines for structured and semi-structured data
  • Familiarity with data lake / delta lake architecture
  • Good understanding of performance optimization techniques in Spark and SQL
  • Experience with version control (Git) and deployment practices
  • Strong problem-solving and analytical skills

Good to Have

  • Experience with cloud platforms (AWS preferably)
  • Exposure to CI/CD pipelines (Jenkins)
  • Knowledge of Delta Lake, Unity Catalog, or similar governance tools
  • Understanding of data quality frameworks / monitoring tools
  • Experience working in Agile / Scrum teams

Location:

This position can be based in any of the following locations:

Chennai

Current Guardian Colleagues: Please apply through the internal Jobs Hub in Workday

Guardian Life

About Guardian Life

Who we are

Guardian makes a difference in the lives of people when they need us most. With over 160 years of stability and fiscal integrity, we are a trusted resource to generations of families and business owners, inspiring well-being and helping build financial confidence.

Today, we stand behind 29 million consumers, helping them prepare and plan for a bright future for themselves and their families. We help business owners care for their employees. And we help people recover and thrive in times of unexpected loss.

As a modern mutual insurance company, we believe in driving value beyond dividends. We invest in our colleagues, are building an inclusive and innovative culture, and are helping to uplift communities through thoughtful corporate impact programs.

What we stand for

In 1860, a community of immigrants joined together to insure and protect their businesses and families. They were guided by powerful ideals that we’ve continued to stand behind and evolved throughout the years: we do the right thing, we believe people count, we courageously shape the future together, and we go above and beyond for the people we serve.

Guardian employees embrace and live by these values every day. They remind us to put people at the heart of all we do so that we can help protect what matters most to you. Want to help bring these values to life? Join us for a rewarding career and the opportunity to shape the future.

Disclosures:

Financial information concerning Guardian as of December 31, 2022, on a statutory basis: Admitted assets = $76.0 billion; liabilities = $67.2 billion (including $55.0 billion of reserves); and surplus = $8.8 billion. Dividends are not guaranteed. They are declared annually by Guardian’s Board of Directors.

Guardian® is a registered trademark of The Guardian Life Insurance Company of America. © Copyright 2023 The Guardian Life Insurance Company of America 2023-156184 Exp. 5/25

Industry
Finance & Insurance
Company Size
5,001-10,000 employees
Headquarters
New York , NY
Year Founded
1860
Social Media