The work
You will help evaluate AI systems on computer science reasoning and coding tasks. The work combines problem design, solution review, Python validation, structured scoring, and clear technical feedback.
- Design rigorous computer science problems based on real professional practice.
- Evaluate AI-generated solutions for correctness, logical soundness, assumptions, constraints, and edge cases.
- Use Python and tools such as NumPy, Pandas, or SciPy to validate calculations and simulations.
- Score complex, multi-step reasoning with structured criteria.
- Improve explanations and reasoning so they follow industry-standard logic.
- Write questions, answers, prompts, and responses, and take part in text generation, coding evaluation, and red-teaming work.
- Help maintain consistent evaluation standards with other advanced experts.
What it pays and takes
This is a part-time contractor role for experienced computer science contributors. The work is project-based rather than permanent, with contributions needed during active project phases.
- Pay: $15 to $40 per hour.
- Time: 20 or more hours per week.
- Location: Open worldwide.
- English: Fluent written English at C1 level or higher.
- Experience: Expert-level computer science knowledge and at least 2 years of applied, research, or teaching experience in computer science or a related technical field.
- Education: A degree in computer science or a closely related field, or equivalent demonstrated expertise.
- Python: Strong proficiency for numerical validation and simulation, including NumPy, Pandas, SciPy, or equivalent tools.
- Evaluation: Ability to assess correctness, assumptions, constraints, edge cases, and solution quality using structured criteria.
- Communication: Clear technical writing and feedback in English.
- Additional experience with MATLAB, R, C, C++, SQL, domain-specific libraries, professional certifications, or international and applied projects is a plus.
How it works
Apply on OpenTrain. The employer reviews applications there.
About AI training work
AI training is the human work behind systems that generate and understand text, code, images, and other data. Experts review examples, test model outputs, and provide clear ratings or corrections so AI systems learn to produce more accurate and useful results. Experienced contributors are paid for the subject knowledge and careful judgment this work requires.