Workiy Inc.

QA Lead – Agentic AI Platform

Workiy Inc.  •  Remote  •  8 hours ago
Apply
Save Job
Mark Applied Mark Interviewed Mark Offered
Hide Job Report Job
AI can make mistakes so check important info. Chat history is never stored.

Job Description


This is a remote position.


Job Description:

We are seeking an experienced QA Lead to drive end-to-end quality assurance for an advanced Agentic AI platform. This role will be responsible for defining quality standards, developing automated evaluation frameworks, leading test strategy, and ensuring release readiness for AI-powered conversational solutions. The ideal candidate will have strong expertise in Python-based test automation, AI/LLM testing, conversational AI validation, and Retrieval-Augmented Generation (RAG) systems. Responsibilities include building evaluation harnesses, creating test plans, executing regression testing, validating data quality, conducting AI safety and prompt injection testing, managing defect triage, and leading a team of QA and Data Testers. The QA Lead will collaborate closely with engineering, product, architecture, and data teams to ensure reliable, scalable, and high-quality AI solutions are delivered while maintaining robust quality gates and release standards.


Requirements


  • QA Leadership

  • Python Test Automation

  • AI/LLM Testing

  • Conversational AI Testing

  • RAG (Retrieval-Augmented Generation) Testing

  • LLM Evaluation Frameworks

  • Golden Dataset & Regression Testing

  • Intent & Context Validation

  • Multi-Turn Dialogue Testing

  • Data Quality Testing

  • Prompt Injection & AI Safety Testing

  • Release Readiness & Quality Gates

  • Defect Management & Test Strategy

  • CI/CD Integrated Testing

  • Team Leadership & Mentoring
Workiy Inc.

About Workiy Inc.

Workiy provides digital solutions and staffing services, utilizing our patented delivery model and a unique and flexible, client-centric approach to tackling technology's biggest challenges. We don’t have “clients” we have partners! Our mission is to enable your talented teams and amazing systems to assist your business in optimally achieving its goals. The Workiy team, brings to the table, more than 20+ years of experience in the technology industry across many verticals in both the private and public sector.

Industry
Unknown
Company Size
11-50 employees
Headquarters
Vancouver, CA
Year Founded
2008
Website
workiy.com
Social Media