agent评测工程师
北京
社招
全职
技术
职位 ID：A133388
职位描述
职位要求
1.Python/Java/Go 至少一门达到工程化水平，具备测试工具自研能力，能独立设计开发自动化测试框架，并根据流程瓶颈自主构建 Mock 服务、数据构造、评测平台等配套工具； 2.接口/UI/CLI 自动化至少掌握两类，熟悉 CI/CD； 3.有 LLM/Agent 评测方法论：评测集构建、判定标准设计（LLM-as-a-Judge 或规则+人工）、工具调用与多轮上下文的量化验证； 4.熟悉性能压测与异常容错测试，能读日志、看链路，定位到模块级； 5.测试视角完整：既能对模型层能力（对话、工具调用、多轮上下文、任务链路）设计评测，也能承接产品层常规功能测试（交互逻辑、权限、数据一致性、异常边界、兼容性），不偏科于模型侧； 6.拥有2 年以上测试开发/质量保障经验，在具备成熟质量体系的团队中完整参与过全流程实践，能清晰阐述流程规范、用例管理、缺陷闭环、发布准入各环节的作用机制，并具备依据团队规模与节奏进行裁剪取舍的判断力； 7.具备体系化的质量保障方法论，能明确质量门禁的设置依据、自动化的投入边界，以及资源受限条件下的优先级判定逻辑； 8.对 AI 时代的质量保障具备独立认知与实践：在输出非确定性场景下定义验收标准、评测集的构建与迭代机制、模型版本升级的回归策略、自动判定与人工标注的配比设计； 9.了解不同企业级 AI 产品的形态与特点（Agent、RAG、智能客服、流程自动化等）及其典型落地场景，理解 B 端客户对 AI 产品的核心质量诉求——结果可控、过程可追溯、失败可兜底，并能据此判断测试重点的优先级。 加分项 1.能独立搭建或改造 CI/CD 流水线（Jenkins/GitLab CI），具备质量门禁设计经验。
投递
ModelBest established in August 2022, is an Artificial Intelligence technology company headquartered in Beijing, China. The company is deeply rooted in the field of general AI, with a focus on the innovation and application transformation of large-scale models. ModelBest boasts a prestigious founding R&D team from the Tsinghua in the AI field. Leveraging numerous cutting-edge technologies in natural language processing, the company is currently constructing a large-scale pre-trained model library and corresponding tools, aiming to standardize the technology and applications of large models.
Based on the capabilities of the CPM series of large models, ModelBest has successfully facilitated the intelligent upgrade and efficiency enhancement of various industries. Moreover, the company has released a multi-modal dialog assistant with hundreds of billions of parameters, “Luca", to the public. Adhering to the core philosophy of "intelligence encompassing all", we look forward to exploring and expanding unknown possibilities with global partners, aiming to ensure AI technology serves humanity for a better life in a safe and inclusive manner, laying a solid foundation for the advent of the AGI world.
OpenBMB (Open Lab for Big Model Base，https://github.com/OpenBMB), founded by ModelBest Inc & TsinghuaNLP, aims to build foundation models and systems towards AGI.
