The work
You will help train and evaluate advanced AI systems by creating realistic software engineering problems and reviewing the code they produce. Your work will turn practical engineering judgment into clear tasks, tests, reference solutions, and evaluation standards.
- Create demanding software engineering tasks for AI model evaluation.
- Write task instructions and define objective standards for correct solutions.
- Build reproducible environments, reference solutions, tests, and evaluation criteria.
- Review generated code and technical deliverables for correctness, robustness, completeness, and reproducibility.
- Find problems with implementation, debugging, dependencies, performance, testing, and environment setup.
- Contribute technical judgment to coding evaluations and benchmark-style tasks.
What it pays and takes
This is a fully remote, part-time contractor role focused on accepted task deliverables. Task availability and completion time may vary.
- Pay: $60 to $120 per hour equivalent, paid per task that meets the project specifications.
- Time: Less than 20 hours per week.
- Location: Remote, open to candidates in the countries specified for this role.
- Language: Fluent in English.
- Experience: Expert-level software engineering ability and prior paid experience training, evaluating, benchmarking, or improving AI systems.
- Technical skills: Strong practical proficiency with Python and professional experience with another language such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.
- Required knowledge: Software design, debugging, testing, version control, and dependency management.
- Helpful evidence: Substantial repositories, open-source contributions, developer tools, systems, publications, benchmarks, or engineering write-ups.
- Helpful experience: AI training tasks, reference solutions, tests, graders, benchmark problems, technical submission reviews, long-horizon or agentic coding evaluations, Linux, Docker, CI/CD, command-line tools, sandboxed environments, or programming competitions.
How it works
You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.
The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 8 screening questions.
About AI training work
AI training is the human work behind systems that learn from examples, including code reviews, model evaluations, and other labeled data. Experienced software engineers are paid for applying careful technical judgment to help make AI outputs more accurate, reliable, and useful.