Job Description

Job Summary

The Senior Technical Lead at HCL will be responsible for leading technical teams and projects related to apache spark, scala, and python. The role involves overseeing the design, development, and implementation of scalable and efficient solutions using these technologies.

Key Responsibilities

1. Lead technical teams in the design and implementation of solutions using apache spark, scala, and python
2. Provide technical expertise and guidance to team members in resolving complex technical issues
3. Collaborate with stakeholders to gather requirements and define project scope
4. Ensure adherence to best practices in coding, testing, and deployment processes
5. Conduct code reviews and performance optimization activities
6. Troubleshoot and debug technical issues to ensure seamless project delivery
7. Stay updated with the latest trends and advancements in apache spark, scala, and python technologies
8. Mentor team members and facilitate knowledge sharing within the team

Skill Requirements

8+ years of Engineering experience primarily in Data Engineering in an agile production environment

High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Oozie, Airflow, NiFi, Scoop), SQL to build Big Data products & platforms

Extensive experience with Databricks, Spark , PySpark or Java + PySprak Processing engine.

Proficiency in, at least, one modern programming language such as Python, Java or Scala

Strong Computer Science fundamentals in object-oriented design, data structures, algorithm design, problem solving, and complexity analysis.

Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it

Working Knowledge in Software Development engineering Paradigms along with Data Engineering.

Relational Databases as well as NoSQL experience

cloud technologies like Databricks/AWS/Azure ( Good to have)

Basic Shell scripting and knowledge of Linux/Unix systems

Experience in designing & developing software at scale

Strong written and verbal English communication skills.

Motivation, creativity, self-direction, and desire to thrive on small project teams

Superior academic record with a degree in Computer Science or related technical field

Other Requirements

1.Relevant certifications in apache spark, scala, or python are a plus