Research Evaluation Specialist (PhD, Researcher, Professor)

You will create rigorous question-and-answer materials used to evaluate advanced AI models. The work focuses on testing reasoning, research judgment, synthesis, and accuracy in your area of expertise.

You will work independently, research authoritative sources, and write questions that cannot be answered through shortcuts or simple oversimplification.

• Create original, difficult questions and defensible answers in your discipline.
• Research primary literature and authoritative references.
• Document citations, reasoning, and supporting evidence.
• Design questions that require advanced reasoning, methodological judgment, or synthesis across sources.
• Test questions against AI systems and improve items that allow shortcuts.
• Write precise and unambiguous question-and-answer pairs.
• Apply reviewer feedback and follow project quality standards.

This is a part-time independent contractor role for researchers and other subject-matter specialists. It is open to candidates in the listed countries and requires strong written English.

• $30 to $60 per hour equivalent, paid per task that meets the project specifications.
• Less than 20 hours per week.
• Part-time, independent contractor work.
• Location: Open to candidates in Algeria, Argentina, Austria, Bahrain, Bangladesh, Belgium, Brazil, Canada, Chile, Colombia, Denmark, Egypt, France, Germany, Ghana, Greece, India, Indonesia, Italy, Japan, Jordan, Kazakhstan, Kenya, Kuwait, Kyrgyzstan, Lebanon, Malaysia, Mexico, Morocco, the Netherlands, Nigeria, Oman, Pakistan, Palestinian Territories, Peru, the Philippines, Portugal, Qatar, Saudi Arabia, Singapore, South Africa, Spain, Sri Lanka, Switzerland, Taiwan, Thailand, Tunisia, the United Arab Emirates, the United States, Uzbekistan, and Vietnam.
• Fluent written English.
• Completed PhD, active doctoral candidacy, or equivalent research experience as a specialist, researcher, or professor.
• Deep subject-matter expertise and experience with scholarly research and primary literature.
• Strong analytical thinking and close attention to detail.
• Experience finding and comparing information from authoritative references.
• Ability to write challenging, original, methodologically sound questions.
• Reliable and self-directed when working remotely.
• Previous AI training or model evaluation experience is helpful but not required.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training is the human work behind systems that learn from examples, including writing and rating model responses and evaluating accuracy. Researchers are paid for this work because their expertise helps create reliable tests and distinguish strong reasoning from weak or unsupported answers.