EXL

Senior Software Engineer - AI Gateway & Agentic AI Solutions

EXL  •  Gurugram, IN (Onsite)  •  2 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Top 5 Responsibilities

  1. Architect and implement the AI Gateway. Design and build an Azure-native LLM gateway in C# providing unified ingress across multiple model providers (Azure OpenAI, Azure AI Foundry models, and others) - covering intelligent routing, fallback/load balancing, rate limiting and token-quota enforcement, semantic caching, and centralized API key/secret management. Reference the LiteLLM feature set as the functional bar to hit.
  2. Build the observability and governance layer. Implement request/response tracing, prompt/completion logging, token and cost metering, and latency dashboards - the Langfuse-equivalent half of the stack - using Azure Monitor, Application Insights, OpenTelemetry, and APIM's native LLM logging/token-metric policies (or a self-hosted Langfuse instance where warranted).
  3. Deliver multiple Agentic AI PoC scenarios. Using C# and the Microsoft Agent Framework, build a portfolio of distinct agentic patterns (single-agent tool use, multi-agent orchestration, human-in-the-loop workflows, RAG-grounded agents) mapped to real client business use cases - not one deep PoC, but several breadth-covering scenarios that demonstrate different capabilities.
  4. Own technical delivery on client engagements end-to-end. Run architecture proposals, hands-on build, live demos, and production-readiness assessments directly with client technical stakeholders; translate ambiguous business asks into scoped, demoable agentic scenarios.
  5. Package the work as reusable engineering assets. Turn the gateway and agent scenarios into templates, SDKs, or IaaC that can be re-deployed across environments rather than rebuilt from scratch each time - even though this is a client-facing role, the artifacts should outlive any single client environment.

Top 5 Required Skills / Background

  1. Deep C#/.NET engineering background (12-15+ years). Production-grade API/service development, async patterns, dependency injection, testing discipline, and comfort operating without a large surrounding platform team.
  2. Hands-on Azure platform expertise. Azure API Management (including GenAI/LLM-specific policies: token-limit, token-metric, semantic-caching), Azure OpenAI / Azure AI Foundry, Azure Monitor / Application Insights, Key Vault, and either Azure Functions/App Service or AKS for hosting gateway services.
  3. Working knowledge of LLM gateway and observability platforms. Familiarity with LiteLLM, Langfuse, or comparable tools (Portkey, Kong AI Gateway) - not to operate them directly, but to translate their proven patterns (routing, fallback, semantic caching, token metering, tracing) into an Azure/C# implementation.
  4. Microsoft Agent Framework experience, or a direct path into it. Hands-on work with MAF, or strong prior experience in Semantic Kernel and/or AutoGen (MAF's predecessors) - multi-agent orchestration, tool/function calling, thread/state management, and ideally MCP/A2A protocol exposure.
  5. Client-facing solution engineering track record. Demonstrated ability to architect, prototype, and present technical PoCs directly to enterprise client stakeholders, defend design trade-offs live, and adapt scope under engagement time pressure.

Top 5 Required Skills / Background

  1. Deep C#/.NET engineering background (12-15+ years). Production-grade API/service development, async patterns, dependency injection, testing discipline, and comfort operating without a large surrounding platform team.
  2. Hands-on Azure platform expertise. Azure API Management (including GenAI/LLM-specific policies: token-limit, token-metric, semantic-caching), Azure OpenAI / Azure AI Foundry, Azure Monitor / Application Insights, Key Vault, and either Azure Functions/App Service or AKS for hosting gateway services.
  3. Working knowledge of LLM gateway and observability platforms. Familiarity with LiteLLM, Langfuse, or comparable tools (Portkey, Kong AI Gateway) - not to operate them directly, but to translate their proven patterns (routing, fallback, semantic caching, token metering, tracing) into an Azure/C# implementation.
  4. Microsoft Agent Framework experience, or a direct path into it. Hands-on work with MAF, or strong prior experience in Semantic Kernel and/or AutoGen (MAF's predecessors) - multi-agent orchestration, tool/function calling, thread/state management, and ideally MCP/A2A protocol exposure.
  5. Client-facing solution engineering track record. Demonstrated ability to architect, prototype, and present technical PoCs directly to enterprise client stakeholders, defend design trade-offs live, and adapt scope under engagement time pressure.
EXL

About EXL

Choosing a digital partner is about more than capabilities — it’s about collaboration and character.

Unrealistic overhauls and off-the-shelf products ignore what matters most — your unique needs, culture, goals, and your legacy data and technology environments.

At EXL, our collaboration is built on ongoing listening and learning to adapt our methodologies. We’re your business evolution partner—tailoring solutions that make the most of data to make better business decisions and drive more intelligence into your increasingly digital operations.

Whether your goals are scaling the use of AI and digital, redesign operating models, or driving better and faster decisions, we’re here to partner with you to help you gain—and maintain—competitive advantage with efficient, sustainable models at scale.

Our expertise in transformation, data science, and change management helps make your business more efficient and effective, improve customer relationships and enhance revenue growth. Instead of focusing on multi-year, resource- and time-intensive platform designs or migrations, we look deeper at your entire value chain to integrate strategies with impact.

We use our specialization in analytics, digital interventions, and operations management—alongside deep industry expertise — to deliver solutions that help you outperform the competition.

At EXL, it’s all about outcomes—your outcomes—and delivering success on your terms. Share your goals with us and together, we’ll optimize how you leverage data to drive your business forward.

For more information, visit www.exlservice.com.

Industry
Consulting & Advisory
Company Size
10,000+ employees
Headquarters
New York, NY
Year Founded
Unknown
Social Media