ModelBest

AI评测平台全栈开发工程师

ModelBest  •  Onsite  •  18 hours ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

AI评测平台全栈开发工程师
北京
社招
全职
技术 - 开发
职位 ID:A26900
职位描述
岗位定位:负责公司级 AI 评测平台的架构设计、全栈研发与持续演进,建设覆盖自动化 Benchmark、人工体验评测、Judge Model 评测、榜单发布和模型晋级门禁的统一评测底座。该岗位需要把分散在算法、数据、训练、推理和产品团队中的评测流程平台化、标准化和可追溯化,支撑模型版本从 Checkpoint 产出到评测报告、Leaderboard 和发布决策的完整闭环。
职责要点:
1、设计并实现评测平台的核心领域模型和平台能力,包括 Benchmark Registry、评测集版本管理、Eval Job 编排、模型 / Checkpoint / 推理服务绑定、Judge Model 管理、人工体验反馈、结果解析归档、报告生成、榜单发布和模型晋级门禁;
2、建设稳定可靠的异步评测任务系统,支持批量评测、任务优先级、资源配额、失败重试、幂等控制、运行日志、结果复现、失败诊断和成本统计;
3、打通训练平台、推理平台、算力平台和数据平台等链路,实现训练产物自动评测、在线模型回归评测、评测数据集管理、算力资源申请和推理服务调用;
4、建设评测结果分析能力,包括指标聚合、版本对比、维度切片、异常样本追踪、人工反馈闭环和可视化报表,帮助算法团队定位模型能力变化,并支撑模型晋级、回滚和对外展示决策。
职位要求
1、本科及以上学历,计算机、软件工程、人工智能、数据科学等相关专业优先;
2、3 年以上全栈开发、平台研发、MLOps、评测系统或复杂业务系统研发经验,能够独立完成核心模块从需求抽象、技术设计到工程落地;
3、熟悉 Python / Go / Java 等至少一种后端语言,并熟悉 React / Vue 或其他主流前端技术栈,具备良好的接口设计、数据建模、异步任务、权限体系、数据库和工程质量意识;
4、理解大模型研发和评测基本流程,熟悉 Benchmark、评测集、自动化评测、人工评测、LLM-as-a-judge、结果分析和模型版本管理等概念;
5、有模型评测平台、实验管理平台、标注平台、Leaderboard / Benchmark 系统、任务编排系统或企业内部研发平台建设经验者优先;
6、具备良好的跨团队沟通能力,能够与算法、训练、推理、数据、产品和业务团队协作,把复杂评测流程沉淀为稳定平台能力。
投递
ModelBest

About ModelBest

ModelBest established in August 2022, is an Artificial Intelligence technology company headquartered in Beijing, China. The company is deeply rooted in the field of general AI, with a focus on the innovation and application transformation of large-scale models. ModelBest boasts a prestigious founding R&D team from the Tsinghua in the AI field. Leveraging numerous cutting-edge technologies in natural language processing, the company is currently constructing a large-scale pre-trained model library and corresponding tools, aiming to standardize the technology and applications of large models.

Based on the capabilities of the CPM series of large models, ModelBest has successfully facilitated the intelligent upgrade and efficiency enhancement of various industries. Moreover, the company has released a multi-modal dialog assistant with hundreds of billions of parameters, “Luca", to the public. Adhering to the core philosophy of "intelligence encompassing all", we look forward to exploring and expanding unknown possibilities with global partners, aiming to ensure AI technology serves humanity for a better life in a safe and inclusive manner, laying a solid foundation for the advent of the AGI world.

OpenBMB (Open Lab for Big Model Base,https://github.com/OpenBMB), founded by ModelBest Inc & TsinghuaNLP, aims to build foundation models and systems towards AGI.

Industry
Biotech & Life Sciences
Company Size
11-50 employees
Headquarters
Beijing, CN
Year Founded
2022
Social Media