The work
You will guide quality decisions for AI annotation and chatbot evaluation projects. The work focuses on applying detailed guidelines consistently, resolving unclear cases, and improving training and data quality.
- Turn complex project guidelines into clear instructions for annotation and review teams.
- Check project setup before production work begins.
- Mentor QA Leads and annotation teams through calibration and feedback.
- Lead calibration sessions and document decisions on escalated cases.
- Support pilot teams testing new task templates.
- Identify high-performing reviewers and analyze audit results.
- Recommend improvements to guidelines and training.
- Compare multiple chatbots and provide final assessments for defined evaluation tasks.
What it pays and takes
This is a fully remote, project-based freelance role supporting AI quality and evaluation work. The role is open worldwide and requires professional fluency in English.
- Hours: 20 or more per week.
- Work arrangement: Remote, part-time contract work.
- Location: Open worldwide.
- Language: Professional fluency in English.
- Experience: Four to six years of overall professional experience, including at least two years leading QA reviewers.
- Strong experience auditing work against detailed guidelines.
- Excellent judgment when handling complex or ambiguous cases.
- Clear communication and documentation skills.
- Experience with annotation, data labeling, AI training data, or multimodal projects is helpful.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site. OpenTrain is free to join and helps people find and build careers in AI training and data-labeling work.
About AI training work
AI training is the human work behind many modern AI systems. People label data, review model outputs, evaluate responses, and document decisions so these systems can learn from clearer, more reliable examples.