The work
You will create challenging computer science problems based on real professional practice and evaluate AI-generated solutions. Your reviews will focus on correctness, logical soundness, assumptions, constraints, edge cases, and the quality of multi-step reasoning.
You will also use Python to check calculations and simulations, improve explanations so they follow industry-standard logic, and apply consistent scoring criteria. The work may include evaluation, question answering, text generation, coding review, and red teaming.
- Design rigorous computer science problems for AI evaluation.
- Validate calculations and simulations with Python tools such as NumPy, Pandas, and SciPy.
- Score complex solutions using structured evaluation criteria.
- Assess assumptions, constraints, edge cases, correctness, and reasoning quality.
- Improve AI-generated reasoning and explanations to meet professional standards.
- Help maintain consistent evaluation standards with other advanced experts.
What it pays and takes
This is a project-based contractor role for an expert contributor. The work is not permanent, and availability is needed during active project phases.
- Pay: $15-$40 per hour, with the listed rate up to $40 per hour.
- Location: Open worldwide.
- Language: Fluent written English at C1+ level or equivalent.
- Availability: About 10-20 hours per week during active project phases; the listing also records a 20+ hour weekly requirement.
- Education: A degree in Computer Science or a closely related field, or equivalent demonstrated expertise.
- Experience: At least 2 years of applied, research, or teaching experience in computer science or a related technical field.
- Technical skill: Strong Python proficiency for numerical validation and simulation.
- Evaluation skill: Ability to assess multi-step solutions for correctness, assumptions, constraints, and edge cases.
- Additional tools such as MATLAB, R, C, C++, SQL, or related libraries are a plus.
- Professional certifications such as SAS or CAP, and international or applied project experience, are a plus.
How it works
Apply on OpenTrain. The employer reviews applications there.
About AI training work
AI training work uses human judgment to create, review, and improve the examples that AI systems learn from. OpenTrain AI is the hiring and contracting organization for this role, connecting qualified contributors with project work that helps improve AI systems.