Job Description

C3 AI is looking for a Senior Quality Engineer, AI and Automation to define and drive the quality strategy for our next generation of AI-enabled products and platforms. In this role, you will design and build scalable automated test frameworks, develop novel approaches for validating AI-generated outputs and agentic workflows, and establish the metrics that keep our products reliable at enterprise scale. You will work at the intersection of software quality engineering and applied AI, ensuring that quality is engineered into every stage of the development lifecycle.

This is a great role for an experienced quality engineer who is passionate about AI and automation and wants to operate with a high degree of autonomy on complex, high-visibility initiatives. You will influence quality strategy across multiple teams, provide technical leadership and mentorship, and partner closely with software engineering, product, and AI stakeholders to raise the bar on how enterprise AI is tested and delivered.

Responsibilities

• Define and execute quality strategies for AI-enabled products and platforms.
• Design, develop, and maintain scalable, automated test frameworks across UI, API, and end-to-end testing.
• Create test plans covering functional, integration, end-to-end, performance, and reliability testing.
• Develop automated validation approaches for AI-generated outputs and agentic workflows, including accuracy, reliability, consistency, and safety.
• Establish quality metrics and dashboards to measure product health and AI system effectiveness.
• Integrate automated tests into CI/CD pipelines and build automated quality gates for AI workflows.
• Identify quality risks early and drive resolution strategies across complex systems.
• Evaluate and implement AI-powered testing tools and methodologies.
• Mentor engineers, promote quality engineering best practices, and drive continuous improvement initiatives across the organization.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Computer Engineering, or a related field.
• 5+ years of experience in Quality Engineering, Test Automation, SDET, or Software Engineering roles.
• Strong programming experience in Java, Python, or a similar object-oriented language.
• Hands-on experience developing and maintaining automated test frameworks, with UI, API, and end-to-end automation testing.
• Experience with modern test automation tools such as Playwright, Selenium, Cypress, or equivalent.
• Experience integrating automated tests into CI/CD pipelines and working with Git.
• Strong understanding of the software development lifecycle, Agile methodologies, and quality engineering best practices.
• Demonstrated ability to analyze complex systems, architect test frameworks, and identify quality risks.
• Excellent written and verbal communication skills.

Preferred Qualifications

• Experience testing AI-powered applications, machine learning systems, LLMs, or AI agents.
• Experience designing evaluation strategies for AI outputs, including accuracy, reliability, consistency, and safety.
• Familiarity with AI evaluation frameworks such as DeepEval, LangSmith, Ragas, or similar tools.
• Experience validating Retrieval-Augmented Generation (RAG) systems and knowledge of prompt engineering and AI model evaluation techniques.
• Experience building automated quality gates for AI workflows and AI-assisted test generation.
• Experience with cloud platforms such as AWS, Azure, or GCP, containerized environments, and observability and monitoring tools.
• Experience mentoring engineers and driving quality initiatives across teams.

C3 AI provides a competitive compensation package and excellent benefits.