Job Description

As AI-powered features become central to automotive vehicles — from ADAS perception and voice assistants to predictive diagnostics and intelligent infotainment — rigorous validation of these systems is critical for successful deployment. We are seeking a Senior AI Validation Engineer to design and implement testing strategies, frameworks, and automated pipelines that ensure the quality, safety, and reliability of deep learning and LLM-based features across vehicle platforms.

This role sits at the intersection of AI/ML engineering and automotive system validation. You will build the tools, datasets, and evaluation methodologies that enable confident delivery of AI-driven automotive solutions.

Key Responsibilities:

• Design and implement validation frameworks for deep learning models (perception, NLP, generative AI) deployed in automotive systems, covering accuracy, robustness, latency, and safety metrics.
• Develop automated test pipelines for LLM-based features, including hallucination detection, response quality evaluation, prompt regression testing, and adversarial input testing.
• Build and curate evaluation datasets and benchmarks tailored to automotive AI use cases (e.g., voice commands, diagnostic Q&A, sensor fusion outputs).
• Create AI-assisted test generation tools that leverage LLMs to automatically produce test cases, test data, and expected-result specifications from system requirements.
• Develop model monitoring and drift detection systems for AI features running in production and test environments.
• Collaborate with system architects to integrate AI model validation into existing test bench infrastructure and CI/CD pipelines.
• Implement automated regression testing for ML model updates, ensuring backward compatibility and performance parity across software releases.
• Analyze test results using statistical methods and ML techniques to identify root causes, failure patterns, and quality trends.
• Work in cross-functional Agile teams spanning AI/ML, embedded software, and system integration disciplines.

Qualifications

Basic Qualifications:

• Bachelor’s degree in computer science, Machine Learning, Data Science, Electrical Engineering, or a related field.
• Minimum of 5 years of experience in ML/AI development, with a minimum of 2 years focused on model evaluation, testing, or validation.
• Strong proficiency in Python and testing/automation frameworks (pytest, Robot Framework, or equivalent).
• Hands-on experience evaluating deep learning models — including metrics design, dataset curation, bias/fairness analysis, and regression testing.
• Experience with LLM evaluation techniques (BLEU, ROUGE, human-in-the-loop evaluation, LLM-as-judge approaches).
• Familiarity with ML experiment tracking and pipeline orchestration tools (MLflow, Weights & Biases, Kubeflow, or equivalent).
• Experience with CI/CD systems (Jenkins, GitLab CI, GitHub Actions) for automated test execution.
• Strong analytical and communication skills with the ability to translate AI validation results into actionable insights for engineering teams.

Preferred Qualifications:

• Master's in Computer Science, Machine Learning, or a related field.
• Experience with simulation-based testing or digital twin environments.
• Familiarity with automotive test toolchains (dSpace, Vector CANoe, NI VeriStand) is a plus but not required.
• Ability to collaborate effectively across time zones with global engineering teams.
• Knowledge of automotive safety standards (ISO 26262, SOTIF/ISO 21448) as applied to AI systems.
• Experience with adversarial robustness testing, out-of-distribution detection, or uncertainty quantification for neural networks.

Stellantis

At Stellantis, we assess candidates based on qualifications, merit, and business needs. We welcome applications from all people without regard to sex, age, ethnicity, nationality, religion, sexual orientation, disability, or any characteristic protected by law. We believe that diverse teams reflect our identity as a global company, enabling us to better address the evolving needs of our customers and care for our future.

Additional Job Posting Information