LLM Response Evaluation Annotator

You will help assess and improve large language models by testing how well they retrieve, analyze, and reason over structured data. You will review generated answers and explain whether they are accurate, logical, complete, and supported by evidence.

• Design challenging prompts that test model reasoning over datasets.
• Evaluate AI-generated answers for factual accuracy, logical reasoning, and completeness.
• Identify hallucinations, inconsistencies, reasoning gaps, edge cases, and other model failures.
• Validate responses against provided datasets and supporting evidence.
• Document clear, evidence-backed explanations while following annotation guidelines and consistent quality standards.

The role is a remote contractor assignment lasting four weeks. Pay details are not provided in the listing.

• Remote assignment open worldwide.
• Available commitments: 20, 30, or 40 hours per week.
• At least four hours of overlap with Pacific Time is required.
• A master's degree or higher in any discipline is required.
• At least three years of professional, research, or teaching experience is required.
• Strong analytical and critical-thinking skills are required.
• Excellent written English communication skills and close attention to detail are required.
• You must be able to validate information against source data.
• Experience with large language models, generative AI, prompt engineering, AI evaluation, data annotation, model testing, CSV files, spreadsheets, or databases is helpful.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training is the human work behind systems that generate and understand text. People write prompts, review model responses, and give detailed feedback so these systems become more accurate, useful, and reliable. Experienced contributors are valued for careful judgment and subject knowledge.