Requirements

Role:
• Own end-to-end quality engineering for agentic
applications, intelligent workflows, APIs, user interfaces, and supporting
platform services.
• Design, develop, and maintain robust
automation frameworks for UI, API, integration, regression, and system-level
testing.
• Define test strategies for AI and agent-based
systems, including workflow validation, response quality, tool usage,
guardrails, reliability, and failure handling.
• Build automated quality gates for CI/CD
pipelines and ensure high-confidence releases through continuous testing.
• Partner closely with software engineers,
product managers, architects, and data/AI teams to ensure quality is built in
from design through production.
• Create meaningful test data, validate edge
cases, identify defects early, and drive root cause analysis through to
resolution.
• Evaluate non-functional characteristics such
as performance, reliability, observability, resilience, and scalability.
• Contribute innovative ideas that improve
testability, developer productivity, release confidence, and engineering
effectiveness.
• Demonstrate strong ownership by proactively
identifying gaps, driving improvements, and mentoring team members on
automation and quality best practices.
About You
• Strong experience in software test automation,
automation framework design, and code quality practices.
• Proven ability to test modern distributed
systems, APIs, event-driven architectures, and user-facing applications.
• Experience validating AI-enabled or agentic
applications, including prompt/workflow validation, output quality assessment,
and scenario-based testing.
• Strong understanding of Agile development,
shift-left testing, continuous integration, and continuous delivery.
• Comfortable working across ambiguity and
applying innovative thinking to solve new and evolving quality challenges.
• Strong communication, collaboration, and
stakeholder management skills, with a bias for action and accountability.
Technical Skills
• Strong programming skills in Java, Python, or
similar languages used for automation and test engineering.
• Hands-on experience with API and UI automation
frameworks such as Playwright, Selenium, Cypress, REST Assured, or equivalent.
• Experience integrating automated tests into
CI/CD pipelines and quality gates.
• Understanding of test approaches for LLMs,
retrieval workflows, agent orchestration, and non-deterministic systems is
highly desirable.
• Familiarity with observability, logging, test
data management, performance testing, and service virtualization is a plus.
Leadership & Mindset
• Innovative thinker who challenges conventional
approaches and introduces better ways to assure quality at scale.
• Strong sense of ownership, accountability, and
follow-through from test strategy through delivery outcomes.
• Able to influence quality culture across teams
and mentor others in modern automation and engineering excellence.

Must-Have

Good to Have

Strong automation engineering background
with hands-on framework design and coding

Experience with LLM evaluation,
prompt/workflow validation, and agent guardrail testing

Hands-on expertise in
Playwright
for UI and end-to-end automation

Exposure to performance, resiliency, and
observability testing for AI systems

Proven experience testing APIs,
integration flows, and CI/CD quality gates

Knowledge of retrieval-based systems,
tool-calling workflows, and non-deterministic test patterns

Experience testing
agentic AI
applications
, including workflow accuracy, tool usage, and failure
handling

Experience with service virtualization,
test data management, and synthetic test environments

Strong programming skills in Java,
Python, or similar languages for automation

Familiarity with cloud-native testing in
distributed/event-driven systems

Ownership mindset, innovative thinking,
and ability to drive quality improvements end to end

Mentoring experience and contribution to
quality engineering best practices across teams