Xiaomi Technology

Agent 评测工程师

Xiaomi Technology  •  Onsite  •  8 days ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

Agent 评测工程师北京社招全职职位 ID:A188605职位描述1. 搭建Agent端到端评测体系,覆盖单轮问答质量、多轮对话一致性、工具调用准确率、任务完成率等维度。
2. 构建和维护评测数据集(Golden Dataset),包括标准问答对、多步推理用例、边界Case、对抗样本等。
3. 开发自动化评测Pipeline,集成到CI/CD流程,实现Agent版本发布前的自动回归测试。
4. 设计Agent安全性评测方案,包括Prompt注入防御、信息泄露检测、权限越界测试。
5. 对标Manus、Genspark等产品的Agent能力,定期进行竞品Benchmark对比评测。
6. 输出评测报告,驱动算法和工程团队持续优化。职位要求1.本科及以上学历,2年以上AI/NLP评测或QA工程经验。
2.熟悉Agent评测框架:RAGAS、DeepEval、TruLens、AgentBench、PromptFoo中至少两种。
3.有评测数据集构建经验,熟悉数据标注流程与质量控制方法。
4.熟悉LangSmith、LangFuse等LLM可观测性工具,能追踪Agent全链路执行日志。
5.了解主流Agent Benchmark(SWE-Bench、WebArena、GAIA、ToolBench等)。
6.精通Python,有自动化测试框架开发经验。
7.了解红队测试(Red Teaming)、对抗性评测方法者加分。
8.有Manus、Genspark、Perplexity等AI Agent产品深度使用与评测经验者优先。 投递
Xiaomi Technology

About Xiaomi Technology

Xiaomi Corporation was founded in April 2010 and listed on the Main Board of the Hong Kong Stock Exchange on July 9, 2018 (1810.HK). Xiaomi is a consumer electronics and smart manufacturing company with smartphones and smart hardware connected by an IoT platform at its core.

Embracing our vision of “Make friends with users and be the coolest company in the users’ hearts”, Xiaomi continuously pursues innovations, high-quality user experience and operational efficiency. The company relentlessly builds amazing products with honest prices to let everyone in the world enjoy a better life through innovative technology.

Xiaomi is one of the world's leading smartphone companies. The company has also established the world’s leading consumer AIoT (AI+IoT) platform,reached 558 million smart devices connected to its platform (excluding smartphones,laptops and tablets) as of September 30 2022. Xiaomi products are present in more than 100 countries and regions around the world. In August 2022, Xiaomi was included in the Fortune Global 500 list for the fourth year in a row, ranking 266th. The company is the fastest-rising Chinese technology conglomerate during the four-year period.

Xiaomi is a constituent of the Hang Seng Index, Hang Seng China Enterprises Index, Hang Seng TECH Index and Hang Seng China 50 Index.

Industry
IT & Software
Company Size
10,000+ employees
Headquarters
Beijing, CN
Year Founded
2010
Social Media