The work
You will create and review board game reasoning tasks that help evaluate AI systems. The work involves turning strategy and tabletop expertise into prompts, benchmarks, rubrics, and quality standards.
You will assess AI-generated answers for correctness, consistency, and reasoning quality, then help improve evaluation datasets and methods.
- Analyze board game scenarios, strategic decision trees, game mechanics, and complex rule systems.
- Evaluate AI answers and identify logical errors, rule violations, and flawed reasoning.
- Create prompts, rubrics, benchmarks, and evaluation guidelines for strategy-focused tasks.
- Review dataset quality and contribute to quality assurance for AI evaluation projects.
- Work with project teams to improve evaluation methods.
What it pays and takes
The listing does not provide a pay rate. The role is part-time, with a commitment of at least 20 hours per week.
- Pay: Not provided in the listing.
- Hours: 20 or more hours per week.
- Location: Open worldwide.
- Language: Fluent English.
- Experience level: Entry level.
- Education: A bachelor's degree in computer science, mathematics, cognitive science, game design, philosophy, economics, or a related analytical field.
- Experience: At least two years of professional or semi-professional experience in board game design, playtesting, tabletop communities, or related strategy-focused environments.
- Knowledge: Logic, probabilistic reasoning, game mechanics, complex rule systems, and at least one of game theory, behavioral economics, decision science, or formal logic.
- Skills: Strong analysis, problem-solving, and written communication.
- Helpful background: Data annotation, AI training, prompt engineering, quality assurance, game or puzzle design, evaluation rubrics, benchmark datasets, or rules-based system analysis.
- Additional useful skills: RLHF, model evaluation, synthetic data generation, LLM benchmarking, Python, SQL, or data analysis tools.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training work is the human side of building artificial intelligence. People create examples, review model answers, and set clear standards so AI systems can reason and respond more reliably.
Experienced contributors are paid for applying specialist knowledge, careful judgment, and subject expertise to this process.