Amazon

Agentic Evaluation Scientist, Amazon Customer Service

Amazon  •  $143k/yr  •  Seattle, WA (Onsite)  •  18 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Amazon's Customer Service (CS) is undergoing a major transformation leveraging agentic AI techniques to investigate why customers experience defects and how to prevent them in the future. We're looking for scientists to help drive the quality and capabilities of these agentic systems.

Amazon's Customer Experience Improvement (CXI Tech) team is looking for a scientists to help drive rigorous development in the agentic era. We seek to identify and eliminate unsatisfactory purchasing experiences that may cause customers to return items or not return to Amazon. This role requires you to re-imagine the way we leverage customer interactions/feedback for defect identification and the actions we take for reliable downstream defect resolution for an improved customer experience throughout the consumer business. Driven by the vision that no defect need ever impact customers twice, we work to develop rigorously tested AI systems to investigate what caused individual defects experienced by customers, and identify the needed changes to ensure they do not occur again. In particular, we are seeking scientists with a proven track record of making efficient use of human annotations to evaluate AI systems.

Come help Amazon ensure the best possible experience for our customers!

Key job responsibilities
- Develop evaluation techniques for agentic solutions that operate at Amazon scale developing techniques to make efficient use of human annotations to establish rigorous performance of agentic systems.
- Develop tools and patterns that improve the performance of agentic investigation systems.
- Create standalone ML powered models when needed to support automated investigation of customer defects.

A day in the life
- Develop techniques to measure and improve the performance of agentic systems, even when the performance of those systems exceed the existing human benchmark
- Interface closely with Engineering and Product partners to align the direction of scientific development with needs of the greater business
- Publish techniques internally and externally (when possible) to promote greater rigor in agentic development

About the team
CXI Tech team owns building technology products to drive defect elimination across Amazon and at scale to safeguard and improve the customer experience across all interactions with Amazon.

Basic Qualifications


- 3+ years of building models for business application experience
- PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
- Experience in patents or publications at top-tier peer-reviewed conferences or journals
- Experience programming in Java, C++, Python or related language
- Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing

Preferred Qualifications

- Experience using Unix/Linux
- Experience in professional software development
- Experience building machine learning models or developing algorithms for business application

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits



USA, WA, Seattle - 142,800.00 - 193,200.00 USD annually

Amazon

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. We are driven by the excitement of building technologies, inventing products, and providing services that change lives. We embrace new ways of doing things, make decisions quickly, and are not afraid to fail. We have the scope and capabilities of a large company, and the spirit and heart of a small one.

Together, Amazonians research and develop new technologies from Amazon Web Services to Alexa on behalf of our customers: shoppers, sellers, content creators, and developers around the world.

Our mission is to be Earth's most customer-centric company. Our actions, goals, projects, programs, and inventions begin and end with the customer top of mind.

You'll also hear us say that at Amazon, it's always "Day 1."​ What do we mean? That our approach remains the same as it was on Amazon's very first day - to make smart, fast decisions, stay nimble, invent, and focus on delighting our customers.

Industry
IT & Software
Company Size
10,000+ employees
Headquarters
Seattle, WA
Year Founded
Unknown
Social Media