Job Description

About InvoiceCloud

InvoiceCloud is a fast-growing fintech leader recognized with 20 major awards in 2025, including USA TODAY and Boston Globe Top Workplaces, multiple SaaS Awards wins for Best Solution for Finance and FinTech, and national customer service honors from Stevie and the Business Intelligence Group. Judges also highlighted our mission to reduce digital exclusion and restore simplicity and dignity to how people pay for essential services, as well as our leadership in AI maturity and responsible innovation. It’s an award-winning, purpose-driven environment where top talent thrives. To learn more, visit  InvoiceCloud.com

Job Details
The SDET – AI Platform is responsible for ensuring the quality, reliability, performance, and safety of generative AI capabilities powering the InvoiceCloud platform. This role builds automated test frameworks, semantic validation tools, and continuous monitoring to support enterprise-grade AI delivery at scale.
You will partner closely with Product, AI Engineering, Data Science, Platform, and Security teams to enable measurable, observable, and responsible AI outcomes. Success in this role is defined by reduced release risk, improved AI correctness, and increased developer velocity through automation and continuous validation.
Success Profile
At InvoiceCloud, success is anchored in our core competencies. These competencies guide how every employee delivers impact across their role
Takes Ownership

• 
  Owns end-to-end AI quality strategy for LLM, RAG, and AI-powered platform features.
• 
  Establishes disciplined validation routines to ensure production readiness and release confidence.
• 
  Ensures compliance with data governance, privacy, security, and responsible AI standards.
• 
  Drives CI/CD quality automation, issue triage, drift detection, and resolution.
• 
  Maintains accountability for AI correctness, observability, and long-term test sustainability.

Drives Efficiency

• 
  Builds automated regression suites, benchmarking tools, and evaluation workflows for AI systems.
• 
  Defines and tracks quality KPIs related to accuracy, performance, latency, safety, and reliability.
• 
  Implements continuous validation and monitoring to reduce manual testing and release friction.
• 
  Partners with engineering teams to improve testability, telemetry, and quality insights across pipelines.
• 
  Uses automation and data-driven signals to accelerate feedback loops and reduce time-to-value.

Innovative

• 
  Designs and implements automation for prompt testing, semantic scoring, and safety checks.
• 
  Uses modern AI frameworks (Azure OpenAI, LangChain, Semantic Kernel, vector databases) to create reusable quality tooling.
• 
  Experiments with new approaches to AI validation, observability, and model evaluation.
• 
  Applies emerging AI quality techniques to strengthen platform scalability and reliability.
• 
  Leverages AI-assisted tools to enhance test coverage, insight generation, and development speed.

Customer Centric

• 
  Validates AI behaviors that directly improve payer, agent, and customer service experiences.
• 
  Measures business impact through accuracy, call deflection, reduced handle time, and outcome quality.
• 
  Advocates for safe, transparent, and trustworthy AI behaviors across the platform.
• 
  Partners with product teams to ensure AI quality aligns with real user needs and expectations.
• 
  Ensures AI systems behave predictably and responsibly in customer-facing scenarios.

Requirements

• 
  Experience in test automation or SDET roles within SaaS or distributed systems
• 
  Hands-on experience validating AI/ML, NLP, or analytics-driven applications
• 
  Strong proficiency in Python and CI/CD automation practices
• 
  Familiarity with Azure OpenAI, LangChain, Semantic Kernel, or vector search technologies
• 
  Knowledge of responsible AI practices, privacy, RBAC, and security testing
• 
  Ability to define quality metrics, automate validation, and support production monitoring
• 
  Strong analytical, communication, and cross-team collaboration skills

InvoiceCloud is committed to providing equal employment opportunities to all employees and applicants. We do not tolerate discrimination or harassment of any kind based on race, color, religion, age, sex, nationality, disability, genetic information, veteran or military status, sexual orientation, gender identity or expression, or any other characteristic protected under applicable laws.

This commitment applies to all aspects of employment, including recruitment, hiring, placement, promotion, termination, layoff, recall, transfer, leave, compensation, and training.

If you require a disability-related or religious accommodation during the application or recruitment process, and wish to discuss possible adjustments, please contact jobs@invoicecloud.com

Click here to review InvoiceCloud’s Job Applicant Privacy Policy.

For recruitment agencies: InvoiceCloud does not accept unsolicited resumes from agencies. Please do not forward resumes to our job aliases, employees, or any other company location. InvoiceCloud is not responsible for any fees associated with unsolicited submissions.