Job Description

Who is Recruiting from Scratch:

Recruiting from Scratch is a premier talent firm that focuses on placing the best product managers, software, and hardware talent at innovative companies. Our team is 100% remote and we work with teams across the United States to help them hire.

Staff AI Engineer

Location

San Francisco, CA

Fully onsite, 5 days per week, Monday–Friday.

Company Stage of Funding

Early-Stage AI Startup

Office Type

On-site — 5 days per week

Salary

$225,000–$350,000 Base

Equity

Competitive Equity

Visa

Open to H-1B and OPT transfers. Other visa situations will be considered on a case-by-case basis.

Employment Type

Full-time

Company Description

This early-stage AI company is building AI agents designed to transform how insurance businesses operate. The company is focused on applying AI to complex insurance workflows, developing agents that can reason through tasks, coordinate actions, and deliver reliable outcomes in real-world business environments.

As AI agents become increasingly capable, building systems that can operate reliably beyond simple prompts and responses is a significant engineering challenge. The company is focused on developing agentic systems that can handle complex workflows, use tools, evaluate their own outputs, and operate over extended periods while maintaining reliability and accuracy.

The engineering team is looking for a Staff AI Engineer who can lead the development of sophisticated AI agent systems, establish evaluation frameworks, and build the infrastructure required to bring agentic applications into production.

This role requires a combination of strong software engineering, practical AI agent experience, and the ability to take products from zero to one in a startup environment. The ideal candidate has worked at a B2B SaaS startup, built AI-powered products from the ground up, and understands how to turn existing LLM capabilities into reliable, production-ready applications.

This is an opportunity for a hands-on technical leader to shape the company's AI architecture, experiment with multi-agent systems, build orchestration infrastructure, and influence the direction of its core AI products.

What You Will Do

1. Build & Scale AI Agent Systems

• Design, build, and deploy sophisticated AI agents that solve complex, real-world insurance workflows.
• Develop agentic applications that use existing large language models rather than focusing on training foundation models from scratch.
• Experiment with multi-agent architectures, reasoning strategies, and feedback-driven approaches to improve agent performance.
• Build agents capable of executing multi-step tasks, interacting with external tools, and maintaining context over extended workflows.
• Develop long-running agent systems that can manage complex tasks reliably.
• Identify failure modes in agent behavior and implement strategies to improve reliability, accuracy, and consistency.
• Translate product requirements into practical AI architectures that can be deployed and maintained in production.
• Balance experimentation with production engineering to ensure that promising approaches become usable product capabilities.

2. Develop AI Evaluation Frameworks

• Build evaluation harnesses to measure the performance, reliability, and quality of AI agents.
• Design systematic approaches to test agent behavior across realistic tasks and edge cases.
• Develop evaluation datasets, test cases, and measurement frameworks that help identify failure modes.
• Establish repeatable methods for comparing prompts, agent architectures, model configurations, and orchestration strategies.
• Investigate how feedback can be used to improve agent reasoning and task completion.
• Develop methods to assess multi-step workflows rather than evaluating only individual model responses.
• Use evaluation results to guide engineering decisions and prioritize improvements.
• Ensure that agent performance is measured against meaningful product and business outcomes.
• Build evaluation infrastructure that supports rapid experimentation without compromising production reliability.

3. Build Agent Orchestration & Infrastructure

• Design and implement orchestration systems that coordinate AI agents, tools, and multi-step workflows.
• Build infrastructure that enables agents to interact with APIs, external services, and business systems.
• Develop and integrate Model Context Protocol (MCP) servers and other tool interfaces where appropriate.
• Create reusable components for agent execution, context management, tool use, and workflow coordination.
• Design systems that support long-running tasks and complex sequences of agent actions.
• Improve the reliability of orchestration through structured error handling, monitoring, retries, and recovery strategies.
• Identify architectural bottlenecks and improve performance, scalability, and maintainability.
• Establish patterns and best practices for building production-grade agentic applications.
• Work across application and infrastructure layers to make agent systems easier to develop, test, and operate.

4. Drive Zero-to-One Product Development

• Take ownership of AI engineering initiatives from early technical exploration through production deployment.
• Work closely with product and engineering stakeholders to identify high-impact opportunities for AI agents.
• Translate ambiguous product requirements into concrete technical approaches and measurable outcomes.
• Prototype new agent capabilities, validate their effectiveness, and turn successful experiments into production systems.
• Make architectural decisions that balance development speed, system reliability, and long-term maintainability.
• Build AI-powered product features that solve real customer problems.
• Iterate based on evaluation results, customer requirements, and observed production behavior.
• Help establish the technical foundations and engineering practices required for the product to scale.
• Take responsibility for the outcomes of systems built, not just individual technical components.

5. Provide Technical Leadership & Engineering Ownership

• Lead technical decisions related to AI agent architecture, orchestration, evaluations, and production deployment.
• Establish engineering standards for building, testing, evaluating, and maintaining AI-powered systems.
• Mentor engineers and help raise the team's technical capabilities.
• Collaborate across product and engineering to identify the right balance between research, experimentation, and execution.
• Take ownership of ambiguous technical problems and drive them toward practical solutions.
• Communicate technical tradeoffs and architectural decisions clearly.
• Help the company develop a repeatable approach to building reliable AI agent products.
• Remain hands-on with implementation while influencing the broader technical direction.
• Contribute to a fast-moving engineering culture that values ownership, iteration, and measurable impact.

Ideal Candidate Background

Experience Requirements

• 8+ years of software engineering, machine learning engineering, or closely related technical experience.
• Staff-level engineering capability, with evidence of leading complex technical initiatives and influencing architecture.
• Experience building AI agents is a must-have.
• Experience taking a B2B SaaS product or major product capability from zero to one at a startup is a must-have.
• Experience building and deploying AI-powered applications using existing LLMs.
• Demonstrated ability to own projects from initial exploration through production deployment.
• Experience working in an early-stage or high-growth startup where priorities evolve quickly.
• Strong individual contribution skills alongside technical leadership and mentorship.
• Undergraduate degree in Computer Science from a top-30 university is a stated requirement.
• Ability to work fully onsite in San Francisco, 5 days per week.

Technical Requirements

• Strong Python engineering skills.
• Deep practical experience building AI agent systems.
• Strong software engineering fundamentals and system design skills.
• Experience building reliable production applications around existing LLMs.
• Ability to design multi-step agent workflows and tool-using systems.
• Experience with agent orchestration, execution flows, and context management.
• Ability to build and maintain evaluation harnesses for AI applications.
• Strong understanding of testing, debugging, monitoring, and improving AI-powered systems.
• Ability to translate experimental AI approaches into maintainable production software.
• Strong technical judgment when making architectural tradeoffs.
• Ability to remain hands-on while leading technical direction.

AI Agents, LLMs & Evaluation Requirements

• Hands-on experience designing, implementing, and deploying AI agents.
• Understanding of the differences between simple LLM integrations and systems capable of executing multi-step tasks.
• Experience with multi-agent systems, agent coordination, or advanced agent workflows.
• Experience evaluating agent behavior across realistic tasks and failure scenarios.
• Ability to design repeatable experiments and measure improvements in agent performance.
• Experience building evaluation harnesses, test suites, or benchmarks for AI applications.
• Understanding of feedback-driven improvement and methods for improving agent reasoning and task completion.
• Experience building systems that use tools, APIs, and external services.
• Ability to identify reliability issues in long-running or complex agent workflows.
• Focus on application-layer AI engineering rather than exclusively on model training or ML research.
• Ability to use existing foundation models effectively without requiring the role to focus on training models from scratch.

Agent Orchestration & Infrastructure Requirements

• Experience building orchestration layers for AI agents or multi-step LLM workflows.
• Ability to design reusable components for tool execution, context management, and workflow coordination.
• Familiarity with MCP servers or similar interfaces for connecting agents to tools and external systems.
• Experience handling errors, retries, state, and recovery in complex workflows.
• Understanding of the operational challenges associated with long-running agent execution.
• Ability to build infrastructure that supports experimentation, evaluation, and production deployment.
• Strong backend engineering capabilities, particularly in Python.
• Ability to design maintainable systems that can evolve as models and agent architectures change.

Startup & Product Engineering Requirements

• Experience at a B2B SaaS startup is essential.
• Demonstrated experience taking a product or major capability from zero to one.
• Ability to operate effectively when requirements are ambiguous and technical direction is still evolving.
• Experience balancing rapid prototyping with production-quality engineering.
• Strong product sense and the ability to connect technical work to customer needs.
• Experience collaborating with product and engineering teams to prioritize high-impact work.
• Ability to build reusable systems without overengineering early-stage products.
• Comfort making decisions with incomplete information and iterating based on feedback.
• A track record of taking ownership beyond narrowly defined engineering tasks.

Soft Skills

• Strong ownership and accountability.
• High technical judgment and problem-solving ability.
• Clear communication and ability to explain complex systems.
• Comfortable making decisions in ambiguous situations.
• Ability to lead through technical expertise and implementation.
• Strong collaboration with product and engineering stakeholders.
• Willingness to mentor engineers and improve team-wide engineering practices.
• Ability to move quickly while maintaining high engineering standards.
• Practical, execution-oriented mindset.
• Strong curiosity about AI agents and their real-world applications.
• Ability to distinguish promising experiments from approaches that are ready for production.
• Comfortable working in a fast-paced, fully onsite startup environment.

Nice-to-Have Qualifications

• Experience building multi-agent systems in production.
• Experience developing MCP servers and tool integrations.
• Experience with long-running agents and complex workflow orchestration.
• Experience building evaluation frameworks for agentic applications.
• Experience applying feedback-driven methods to improve agent performance.
• Experience with AI applications in insurance or other complex, regulated business domains.
• Experience establishing AI engineering practices at an early-stage startup.
• Experience mentoring engineers and influencing technical strategy across a team.

Compensation & Benefits

• Base salary: $225,000–$350,000.
• Equity: Competitive equity package.
• Visa: H-1B and OPT transfers supported; other situations considered case by case.
• Work model: Fully onsite in San Francisco, 5 days per week.
• Employment: Full-time.
• Opportunity to own core AI agent architecture and technical direction.
• Hands-on leadership in building production-grade AI systems.
• Opportunity to shape a zero-to-one AI product in an early-stage startup.

Why Join

• Build production AI agents: Work on sophisticated agentic systems that solve real business problems.
• Own zero-to-one engineering: Help take AI capabilities from early experimentation to production deployment.
• Shape AI architecture: Influence multi-agent systems, orchestration, evaluations, and the broader technical direction.
• Solve difficult engineering problems: Build reliable systems for multi-step tasks, tool use, and long-running workflows.
• Define evaluation standards: Establish the frameworks used to measure and improve agent quality.
• Stay hands-on: Lead technically while continuing to design, implement, and ship production software.
• Work in a startup environment: Take meaningful ownership and help establish engineering practices as the company grows.