MiniMax

高级爬虫工程师

MiniMax  •  Onsite  •  15 days ago
Apply
AI can make mistakes so check important info. Chat history is never stored.

Job Description

高级爬虫工程师北京、上海社招全职研发 - 大数据职位描述1、负责爬虫策略与方案制定,爬虫系统架构设计与开发;
2、负责网站、APP等大规模文本、图片数据抓取,爬虫核心算法优化,提升爬虫效率和质量;
3、进行多平台信息分析,参与企业数据建设。职位要求1、3年以上开发经验(爬虫),计算机相关专业优先;
2、有过爬虫平台化建设经验(0-1)优先;
3、至少熟悉并使用过一种主流爬虫架构,如Scrapy、Gocolly、Webmagic等;4、有分布式爬虫或熟悉app逆向、反编译经验者优先;
5、熟练掌握Python/Go/JavaScript等至少一种语言;熟练使用Go语言者优先;
6、熟悉HTTP、TCP等网络协议,精通httpClient/jSoup/xpath/正则表达式/验证码加密处理/代理池等爬虫相关技术;掌握fiddler、charles等抓包工具;
7、熟悉Mysql、Nosql数据库设计,熟悉Redis技术和ElasticSearch;了解Hadoop、HBase、HDFS、Kafka等大数据相关知识。 投递
MiniMax

About MiniMax

MiniMax is a leading global technology company and one of the pioneers of large language models (LLMs) in Asia. Our mission is to build a world where intelligence thrives with everyone.

MiniMax develops proprietary LLMs across various modalities, including a trillion-parameter MoE model, a speech model with low latency and native support for major Asian languages, and a state-of-the-art text-to-speech and text-to-video models. Experience it now at https://hailuoai.com/

Leveraging these multi-modality general-purpose models, the MiniMax API Platform offers enterprises and developers secure, flexible, and reliable API services, enabling the rapid deployment of AI applications.

Industry
IT & Software
Company Size
51-200 employees
Headquarters
Singapore, SG
Year Founded
2022
Social Media