Job Description

We are looking for an AI MLOps Engineer (Full Stack) to support the deployment, monitoring, governance, and ongoing operations of AI and Generative AI solutions within an enterprise AI platform.

The successful candidate will be responsible for implementing MLOps pipelines, monitoring AI model performance, developing evaluation frameworks, and maintaining AI security and governance controls.

You will also support agentic AI operations, including agent monitoring, risk management, security testing, and compliance, to ensure AI systems operate reliably, securely, and responsibly.

Key Responsibilities

• Implement and maintain governance controls for AI models and services to ensure secure, reliable, and compliant AI operations.
• Develop and maintain AI model monitoring solutions covering accuracy, model drift, hallucinations, latency, cost, and usage.
• Design and execute AI model evaluation processes, including benchmarking, regression testing, validation datasets, and human-in-the-loop reviews.
• Implement audit logging and traceability of AI model inputs, outputs, and usage to support governance and compliance requirements.
• Configure and maintain AI security guardrails, including content filtering, Personally Identifiable Information (PII) detection, data redaction, and prompt injection protection.
• Implement and manage Amazon Bedrock Guardrails and other AI safety mechanisms.
• Ensure AI applications comply with relevant data security, privacy, PDPA, and Responsible AI requirements.
• Develop and maintain MLOps pipelines for model deployment, prompt versioning, release management, rollback, and lifecycle management.
• Build full-stack monitoring dashboards and reporting tools to provide visibility into AI performance, operational risks, and compliance status.
• Configure alerts and incident response procedures to detect and address AI service anomalies, performance degradation, and security issues.
• Collaborate with AI Platform Engineers, AI/ML Engineers, and DevSecOps teams to integrate governance and security controls into AI development workflows.
• Conduct security checks, risk assessments, and compliance reviews for AI models, applications, and supporting infrastructure.
• Maintain technical documentation, model cards, governance standards, operating procedures, and compliance records.
• Implement agent-level security controls, including tool allow-lists, permission restrictions, action limits, and human approval for high-impact operations.
• Configure agent execution controls, including loop limits, step limits, spending thresholds, and emergency stop mechanisms.
• Evaluate AI agent behaviour, including task completion, tool usage accuracy, response groundedness, and execution reliability.
• Conduct regression testing and behavioural assessments before deploying changes to AI agents or workflows.
• Perform AI red-teaming and security testing to identify vulnerabilities such as prompt injection, excessive agency, data leakage, and tool misuse.
• Implement end-to-end observability and audit trails for AI agents, including prompts, tool calls, outputs, approvals, and execution traces.
• Develop monitoring dashboards and alerts aligned with service-level indicators, service-level objectives, and AI risk thresholds.
• Maintain compliance evidence for ICT security requirements, including IM8, SSP, PDPA, and Responsible AI standards.
• Support architecture reviews, security assessments, and AI governance initiatives.
• Work closely with Senior AI Platform Engineers and Solution Architects to implement AI security controls, governance standards, and operational best practices.

Requirements

• Diploma or Degree in Computer Science, Data Science, Information Technology, or a related discipline.
• Minimum 2 years of relevant experience in software engineering, MLOps, DevOps, data engineering, or AI operations.
• Good programming skills in Python.
• Working knowledge of JavaScript/TypeScript for developing dashboards, web applications, or internal tools.
• Experience with AI model monitoring, evaluation, or observability tools such as Amazon CloudWatch, SageMaker Model Monitor, MLflow, or Langfuse.
• Understanding of LLM evaluation techniques, including groundedness, faithfulness metrics, LLM-as-judge, and red-teaming.
• Familiarity with Amazon Bedrock capabilities such as Guardrails, Model Evaluation, and model invocation logging.
• Experience with CI/CD pipelines and version control systems such as Git.
• Knowledge of container technologies such as Docker and Kubernetes.
• Good understanding of data security, privacy, AI governance, and Responsible AI principles.
• Understanding of agentic AI risks, including excessive agency, prompt injection, unauthorised tool usage, and data leakage.
• Familiarity with human-in-the-loop controls and AI agent evaluation techniques.
• Understanding of AI observability and tracing tools such as Langfuse or OpenTelemetry.
• Knowledge of government ICT security requirements, including IM8 and SSP, and their application to AI guardrails, audit logging, and monitoring.
• Strong analytical skills, attention to detail, and a risk-aware approach to engineering.
• Comfortable working in Agile development environments and collaborating with cross-functional teams.
• Good communication, troubleshooting, and technical documentation skills.

Added Advantages

• Familiarity with Singapore's Model AI Governance Framework, AI Verify, IM8, and PDPA.
• Experience with Government Commercial Cloud (GCC) or security assessments for government projects.
• AWS certifications such as Machine Learning Engineer Associate, AI Practitioner, or Cloud Practitioner.