Sports Domain Expert for LLM Evaluation

You will help evaluate large language models, or AI systems that generate text. Your work will focus on sports topics such as leagues, athletes, tournaments, rules, statistics, and analytics.

You will create prompts, review generated answers, find factual and reasoning problems, and give evidence-based feedback. You will also help build benchmark datasets and adversarial test cases, which are difficult examples designed to reveal model weaknesses.

• Create advanced prompts covering global sports, leagues, athletes, tournaments, rules, statistics, and analytics.
• Rate AI-generated responses for factual accuracy, reasoning, completeness, and nuance.
• Find hallucinations, logical inconsistencies, outdated information, and edge cases.
• Develop benchmark datasets and adversarial test cases.
• Provide reliable references and maintain careful annotation quality and documentation.
• Work with AI researchers on sports-focused model evaluation.

This is a remote contractor assignment for an eight-week engagement. The work requires strong sports knowledge, excellent written English, research ability, and objective reviews supported by evidence.

• 20 or more hours per week.
• Open worldwide, with at least four hours of Pacific Time overlap required.
• Master's degree or higher required.
• Strong knowledge of sports, including major leagues, teams, athletes, tournaments, rules, statistics, and key developments in one or more sports.
• Excellent written English, research, analytical, and attention-to-detail skills.
• Three or more years of experience in sports journalism, media, content, research, analysis, reporting, or a related field preferred.
• Experience with large language models, generative AI, prompt engineering, or AI evaluation preferred.
• No pay rate is provided for this role.

Apply on OpenTrain with your resume, then complete the application on the hiring site. OpenTrain is where you find this role and start the application; it does not employ or pay contributors for the assignment.

AI training is the human work behind systems that generate and understand content. People write examples, review model responses, and identify errors so AI systems can perform more reliably, and specialized knowledge helps contributors assess difficult topics accurately.