Domain Contributor Evaluation Specialist

You will help improve advanced AI systems by creating difficult, domain-specific questions and scenarios, then reviewing the responses they produce. Your work will focus on finding reasoning problems, factual errors, weak sources, and missing details.

You will use authoritative primary sources to check claims and provide clear feedback. You will follow detailed task guidelines and evaluation rubrics while delivering written and verbal evaluations.

• Design nuanced questions and scenarios that test the limits of AI models.
• Assess generated responses for accuracy, reasoning, source quality, and important nuances.
• Explain missed reasoning, factual errors, source problems, and other weaknesses.
• Create clear written and verbal deliverables based on evaluation guidelines.

This is an intermediate-level, part-time contractor role for people with experience in AI model training, annotation, reinforcement learning from human feedback, or model evaluation. Deep expertise in at least one relevant subject area is also required.

• Pay: $30 to $60 per hour equivalent, paid per task that meets the project specifications.
• Time: Less than 20 hours per week.
• Location: Open to candidates in the listed countries across North America, South America, Europe, Africa, Asia, and the Middle East.
• Language: Professional fluency in English.
• Expertise: History, archival or library science, law, finance, journalism, technology, or a related field.
• Research: Advanced research skills and experience checking complex claims against authoritative sources.
• Communication: Excellent written and verbal communication, with careful attention to detailed instructions.
• Helpful background: Adversarial task design, graduate-level research, experience at a Fortune 100 company, or professional fluency in another language.

Apply on OpenTrain with your resume and then complete the application on the hiring site.

AI training is the human work behind artificial intelligence. People create examples, review model responses, and explain what is accurate or useful so AI systems can improve; people with strong subject knowledge are needed for difficult evaluations and source checking.