Responsibilities

Architectural Leadership

• Own end-to-end architecture for the engagement, from agent orchestration through data integration to deployment.

• Define architectural standards and ensure consistency across backend, UI, data, and infrastructure layers.

• Evaluate trade-offs between latency, cost, accuracy, and maintainability, and document the decisions.

• Set long-term technical direction in coordination with the Delivery Lead and Upstart13 leadership.

• Evaluate and selectively introduce emerging patterns where they measurably reduce risk, cost, or time-to-value, balanced against the client's production and compliance constraints.

• Design for multi-tenant deployment and repeatable team onboarding from the outset, so new teams and semantic models can be brought on without linear growth in architect effort.

• Maintain architectural decision records and standards documentation sufficient for continuity, audit, and onboarding of new engineers.

Agentic AI and Data Integration

• Design the agentic surface: tool calling, retrieval, multi-turn orchestration, and state management.

• Architect retrieval and integration patterns against the client’s semantic models and broader data estate.

• Define evaluation and observability strategies for LLM components, covering regression detection, hallucination tracking, and cost telemetry.

• Design query routing and the intelligence layer that directs questions across a growing set of semantic models as additional teams are onboarded.

• Architect the knowledge-base ingestion and metadata/semantic enrichment pipeline that grounds and improves the agent's accuracy over time.

Cloud, Security, and Multi-Tenant Deployment

• Architect the engagement’s cloud environment across identity, networking, compute, storage, and monitoring.

• Partner with the security team on threat modeling and the security, accessibility, privacy, and responsible AI reviews required for production deployment.

• Ensure high availability, resilience, and secure configurations across the engagement’s footprint.

• Own the environment strategy (dev, test/QA, and production isolation) to support safe iteration and parallel client testing.

• Own the identity and consent architecture (Entra app registrations, admin consent, and on-behalf-of / managed-identity flows) and drive resolution of access blockers with the client.

DevOps and Delivery Enablement

• Drive adoption of CI/CD, infrastructure-as-code, and evaluation-in-pipeline for LLM components.

• Establish templated infrastructure and automated onboarding, so additional teams and tenants can be provisioned with minimal incremental effort.

• Improve observability and operational excellence across the engagement.

• Define and uphold deployment, release, and incident response practices.

Hands-On Contribution

• Review and contribute to agent orchestration, tool integrations, and retrieval pipelines.

• Review and contribute to infrastructure and pipeline definitions.

• Participate in complex debugging and production issue resolution.

• Contribute directly to the overall code base where needed.

Influence and Growth

• Identify capability gaps within the engagement team and surface them to leadership.

• Mentor engineers on agentic AI patterns, Microsoft platform conventions, and architectural reasoning.

• Influence decisions across both Upstart13’s team and the client’s engineering organization without formal authority.

• Represent Upstart13 in technical reviews with senior client stakeholders, including QBRs.

Qualifications

Technical Skills:

• 10+ years of professional experience in software engineering, including architectural responsibilities on large-scale production systems.

• Strong hands-on experience with backend and API development using Python, including modern async backends (FastAPI or equivalent), and React for the frontend.

• Demonstrated experience designing and shipping agentic AI systems in production, including tool calling, multi-turn orchestration, and evaluation.

• Direct experience with Azure AI services (AI Foundry and Azure AI Search) and with agent orchestration in Python, whether via a Microsoft-aligned framework (Semantic Kernel, AutoGen, or Microsoft Agent Framework) or an equivalent custom orchestration approach.

• Working knowledge of Microsoft Fabric, Power BI semantic models, and DAX, sufficient to design integrations with them, not necessarily to author advanced models from scratch.

• Strong knowledge of Azure architecture Container Apps, Container Registry, Entra ID, RBAC, networking, Key Vault, Application Insights.

• Experience with Microsoft Entra applications, including admin consent and Microsoft Graph permission scoping.

• Experience with Azure DevOps for source control, pipelines, and work tracking.

• Experience with Bicep or ARM templates for Infrastructure-as-Code.

• Experience implementing CI/CD pipelines and DevOps best practices.

• Strong understanding of cloud security, RBAC, and governance, including least-privilege design for service principals and managed identities.

Soft Skills:

• Strong ability to set technical direction and influence decision-making across both internal teams and senior client stakeholders.

• Capacity to evaluate complex AI and data systems and design pragmatic solutions under constraints (security review, compliance posture, client-imposed tooling).

• Strategic thinking with hands-on execution capability.

• Proactive ownership with a bias toward shipping.

• Excellent written and verbal communication, including the ability to present architectural decisions to non-technical executives.

• Comfort operating in a client-embedded environment where the client’s preferences and constraints shape the solution.

• Comfortable navigating ambiguity and creating clarity. Moves forward decisively when requirements, constraints, or stakeholders are still in flux.

Bonus Skills:

• Experience with Chainlit, Streamlit, or similar Python AI UI frameworks.

• Experience with LLM evaluation tooling (Azure AI Foundry evaluations, promptfoo, Ragas, or equivalent).

• Familiarity with responsible AI review processes (content safety, accessibility, privacy impact).

• Azure certifications, particularly AZ-305 and AI-102.