Job Description

Role Title: Humanities Evaluation Specialist

Role Type: Contractor

Location: Remote

We are engaging Humanities Evaluation Specialists to contribute to a customer's innovative AI project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Research & source triangulation

• Attention to detail
• Written precision
• Self-direction & reliability
• Analytical thinking
• Humanities studies & experience

Scope of Work

• Develop original, high-difficulty question-and-answer pairs that challenge advanced AI models on interpretation, argument, context, and analytical reasoning within the humanities.
• Research, verify, and provide authoritative citations to support accurate answers and reasoning, referencing primary texts and reputable sources.
• Design prompts that demand deep analytical or interpretive insight rather than surface-level recall.
• Test questions against AI models, iterating to increase difficulty and sharpen clarity while ensuring correctness.
• Write with precision and attention to detail, crafting unambiguous questions and well-supported answers.
• Apply feedback from reviewers to refine submissions and maintain alignment with project guidelines and quality standards.

Preferred Qualifications

• Advanced academic background or professional experience in a humanities discipline such as history, literature, philosophy, linguistics, classics, art history, or journalism.
• Exceptional analytical writing, close-reading, and question formulation skills.
• Demonstrated meticulousness and ability to document reasoning and sources clearly.
• Experience producing challenging, nuanced questions that probe for understanding beyond surface-level facts.
• Self-directed and highly reliable in delivering project tasks independently within a remote context.
• Familiarity with AI model evaluation or training is a plus, though not required.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.