Job Description

We are recruiting for an AI & Solution Testing Analyst in the Legal, Risk Management and Compliance (LRMC) team.

Roles & responsibilities

We are recruiting for an AI & Solution Testing Analyst in the Legal, Risk Management and Compliance (LRMC) team.

• Test Planning & Execution: Develop and execute structured test plans, scenarios and cases for AI-enabled applications, low-code agents and digital solutions.
• AI Output Evaluation: Assess agent responses for accuracy, relevance, consistency, safety, reliability and alignment with approved requirements and expected outcomes.
• Solution Testing: Perform functional, regression, integration, system, usability and edge-case testing across data engineering solutions, Power Platform applications, web applications and reporting solutions.
• Defect Management: Document defects with clear evidence, severity and reproduction steps; track remediation and verify fixes through retesting.
• Requirements Traceability: Review business and technical requirements, define acceptance criteria and maintain traceability between requirements, test coverage and results.
• Automation & Reusability: Identify opportunities for test automation and establish reusable test assets, checklists, standards, metrics and reporting templates.
• Collaboration & Quality Governance: Work with developers, product owners, data engineers and business stakeholders to support release readiness and continuous quality improvement.
• Documentation & Reporting: Maintain test evidence, defect dashboards, quality metrics, release recommendations and knowledge-transfer documentation.

Mandatory technical & functional skills

• 2-3 years of hands-on experience in software testing, quality assurance and defect management for enterprise digital solutions.
• Experience testing AI-enabled applications, copilots or low-code agents, including structured evaluation of outputs for accuracy, relevance, consistency and reliability.
• Strong knowledge of functional, regression, integration, system, usability, end-to-end and edge-case testing techniques.
• Ability to create test strategies, test plans, test scenarios, test cases, test data, acceptance criteria and requirements traceability.
• Experience documenting, prioritizing and tracking defects through resolution, retesting and closure using a test or work-management tool.
• Working knowledge of testing solutions built with Microsoft Power Platform, including Power Apps, Power Automate and Copilot Studio.
• Ability to validate data engineering pipelines and transformations using SQL, including source-to-target checks, reconciliation and data-quality testing.
• Experience testing web applications, APIs and system integrations, including browser compatibility and negative testing.
• Experience validating reports and dashboards for data accuracy, calculations, filters, refresh behavior and usability.
• Strong analytical thinking, attention to detail, documentation and stakeholder communication skills in an Agile delivery environment.

Preferred technical & functional skills

• Experience evaluating AI-enabled applications and low-code agents through prompt testing, grounded Ness checks, hallucination detection, response consistency, safety validation and human-in-the-loop testing.
• Hands-on experience testing Microsoft Power Platform solutions, including Power Apps functionality and usability, Power Automate workflow execution and exception handling, and Copilot Studio agent behavior.
• Experience validating Databricks and data engineering solutions through source-to-target checks, transformation validation, reconciliation, data-quality rules, job execution and failure scenarios.
• Experience testing web applications and REST APIs using tools such as Playwright, Selenium or Postman, including UI, browser compatibility, integration, negative and end-to-end testing.
• Experience validating Power BI reports and dashboards for data accuracy, calculations, filters, drill-through behavior, security, refresh results, performance and usability.
• Exposure to automated regression testing, CI/CD quality checks and defect tracking using Azure DevOps, GitHub or similar test and work-management tools.
• Knowledge of performance, accessibility, security and user-acceptance testing, with the ability to create reusable test packs, scorecards, quality metrics and release-readiness criteria.