Job Description

The position is described below. If you want to apply, click the Apply Now button at the top or bottom of this page. After you click Apply Now and complete your application, you'll be invited to create a profile, which will let you see your application status and any communications. If you already have a profile with us, you can log in to check status.

Need Help?

If you have a disability and need assistance with the application, you can request a reasonable accommodation. Send an email to Accessibility (accommodation requests only; other inquiries won't receive a response).

Regular or Temporary:

Regular

Language Fluency: English (Required)

Work Shift:

1st shift (United States of America)

Please review the following job description:

***This position is 5 days a week in the Charlotte Office***

The Senior AI QA Engineer owns the quality validation of AI-enabled applications, agent workflows, and non-deterministic outputs in production-style enterprise environments.

This role combines the rigor of senior software quality engineering with the specialized evaluation discipline required for AI systems where correctness, completeness, hallucination risk, bias, safety, and reliability must all be measured explicitly.

This is a hands-on role focused on testing AI-powered products, agent-integrated workflows, prompt-driven solutions, and enterprise automations. The engineer designs and executes quality strategies spanning functional testing, regression testing, API validation, UI verification, output scoring, hallucination detection, evaluation dataset design, drift monitoring, and deployment gate readiness for agentic systems.

The role operates within the Forge model: production-first, governance-aware, and evidence-driven. Daily work includes defining test approaches for AI workflows, building automated test assets, validating prompts and outputs, running scenario-based and adversarial checks, partnering with engineering to remediate defects, and ensuring AI-enabled capabilities meet quality, security, and operational standards before release.

***For this opportunity, Truist will not sponsor an applicant for work visa status or employment authorization, nor will we offer any immigration-related support for this position (including, but not limited to H-1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN-1 or TN-2, E-3, O-1, or future sponsorship for U.S. lawful permanent residence status.)***

ESSENTIAL DUTIES AND RESPONSIBILITIES
Following is a summary of the essential functions for this job. Other duties may be performed, both major and minor, which are not mentioned below. Specific activities may change from time to time.

• Design and execute quality strategies for AI-powered applications, agents, prompts, and workflow automations, covering functional behavior, output quality, regression risk, and production readiness.
• ValidateAI outputs for accuracy, completeness, consistency, policy adherence, and business usefulness using defined scoring rubrics, golden datasets, and repeatable evaluation methods.
• Build andmaintainmanual and automated test assets for UI, API, workflow, and agent-integrated solutions, including tests for prompts, tools, memory behavior, and non-deterministic outputs.
• Conduct hallucination, drift, and quality degradation testing for agentic systems and AI-enabled workflows before and after model, prompt, or data changes.
• Partner with engineering teams to investigate failures, isolate root causes, and drive remediation for defects affecting AI output quality, system behavior, or release confidence.
• Define and enforce quality gates for AI features and deployments, including acceptance thresholds for output quality, test coverage, observability, and release readiness.
• Support evaluation of tool-calling behavior, workflow branching, human-in-the-loop steps, and failure handling across complex AI or agentic user journeys.
• Create andmaintaintest data, scenario sets, and benchmark suites that reflectreal businessconditions, edge cases, and regulated enterprise requirements.
• Participatein sprint planning, grooming, demos, and retrospectives to ensure quality considerations are built into design, acceptance criteria, and release commitments from the start.
• Document defects, findings, evaluation results, and release-readiness evidence clearly enough for engineering, product, and governance stakeholders to act on quickly.
• Collaborate with AI security, product, engineering, and operations teams to ensure deployed solutions meet Forge standards for safety, quality, traceability, and auditability.
• Continuously improve AI QA methods, automation coverage, benchmark datasets, and operational test practices as the platform and use cases scale.

QUALIFICATIONS
Required Qualifications:
The requirements listed below are representative of the knowledge, skill and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

• Systems thinker who sees the big picture and the interconnection amongst its parts.
• Ten year’s experience in corporate environment leading key initiatives
• Bachelor’s degree or equivalate experience
• Broad functional knowledge in applied enterprise information security technologies and identity management
• Excellent Communication
• Previous experience in leading complex IT projects
• Previous experience in leading infrastructure and wide-scale rollouts.

Additional Requirements:

• 5+ years of QA or software test engineering experience, including experience with enterprise application testing and production release quality.
• Demonstratedexperience testing AI-enabled applications, machine learning workflows, LLM-powered features, or non-deterministic output systems.
• Strong understanding of software quality disciplines including functional testing, regression testing, API testing,defectlifecycle management, and release validation.
• Experience designing or applying scoring frameworks, expected-output datasets, or benchmark-based evaluation methods for AI quality validation.
• Hands-on experience with automated testing tools and CI/CD-aligned test execution for enterprise software delivery.
• Ability to assess AI outputs for hallucination, inconsistency, incompleteness, unsafe content, or task failure using structured and repeatable methods.
• Strong communicationand documentation skills with the ability to translate ambiguous AI quality failures into actionable findings for engineers and product teams.
• Experience working within enterprise governance, security, and deployment controls where evidence-based release readiness isrequired

PREFERRED QUALIFICATIONS

• Experience testing agentic systems, multi-step orchestration workflows, or tool-calling solutions that combine prompts, APIs, and business logic.
• Experience with AI evaluation frameworks, prompt testing approaches, benchmark sets, or enterprise agent quality tooling.
• Experience with Azure, Microsoft Copilot / Copilot Studio, enterprise workflow automation, or AI-enabled productivity platforms.
• Experience with API, UI, and workflow testing across cloud-native or event-driven enterprise systems.
• Experience in financial services, cybersecurity, risk, compliance-heavy environments, or other regulated industries.
• Familiarity with observability metrics, telemetry, model change validation, and post-release quality monitoring for AI-enabled capabilities.
• Experience collaborating with AI security or adversarial testing teams on release quality, safety, and control validation.

General Description of Available Benefits for Eligible Employees of Truist Financial Corporation: All regular teammates (not temporary or contingent workers) working 20 hours or more per week are eligible for benefits, though eligibility for specific benefits may be determined by the division of Truist offering the position. Truist offers medical, dental, vision, life insurance, disability, accidental death and dismemberment, tax-preferred savings accounts, and a 401k plan to teammates. Teammates also receive no less than 10 days of vacation (prorated based on date of hire and by full-time or part-time status) during their first year of employment, along with 10 sick days (also prorated), and paid holidays. For more details on Truist’s generous benefit plans, please visit our Benefits site Depending on the position and division, this job may also be eligible for Truist’s defined benefit pension plan, restricted stock units, and/or a deferred compensation plan. As you advance through the hiring process, you will also learn more about the specific benefits available for any non-temporary position for which you apply, based on full-time or part-time status, position, and division of work.

Truist is an Equal Opportunity Employer that does not discriminate on the basis of race, gender, color, religion, citizenship or national origin, age, sexual orientation, gender identity, disability, veteran status, or other classification protected by law. Truist is a Drug Free Workplace.

EEO is the Law E-Verify IER Right to Work