Job Description

Job Summary

Key Responsibilities

AI Performance Manager

• Full accountability for agent performance by defining business outcome metrics, operational boundaries, and stability requirements in close collaboration with process owners and business leads.
• Design comprehensive business and technical testing strategies to validate agent effectiveness and reliability from poc, development through to production.
• Lead the deployment and monitoring of AI agents on Google Cloud Platform (GCP), ensuring smooth transitions and tracking on safety and stability metrics, business KPIs to guarantee ongoing stability.
• Act as the ultimate gatekeeper for quality, aligning final sign-off on performance and stability with sponsors before any agent/agentic system is released into the operational environment.

Skill Requirements

Experience: 5-7+ years in a senior technical role (e.g., Data Science Tech Lead, Solution Architect) with at least the last 2 years focused on generative AI, Agentic AI and Agents / Agent Systems. Proven success in defining performance metrics, creating testing frameworks, and managing cloud deployments is essential.

Key Skills:

Deep understanding of agentic/generative AI and the complete agent development lifecycle.

Expertise in designing business and technical testing concepts for AI systems.

Hands-on experience with deploying and managing applications on Google Cloud Platform (GCP).

Experience in Observability, AI output validation, logging, data lineage, latency with working experience in GCP (vertex AI) and Agentic AI ecosystem.

Other Requirements