The Work
You will evaluate AI chat and search tools under consistent conditions. Your reviews will help improve the quality, accuracy, helpfulness, and safety of model training data.
The role combines response rating, prompt writing, data submission, and quality review. You will work with detailed guidelines and help keep evaluations consistent across contributors.
- Assess AI-generated responses using detailed grading rubrics.
- Rank and annotate model outputs for quality, accuracy, helpfulness, and safety.
- Write, refine, and test prompts across different user goals and edge cases.
- Find rating differences, resolve unclear guidelines, and correct evaluation issues.
- Record findings and submit evaluation datasets through AI training tools.
- Take part in quality reviews and collaborate with trainers and other contributors.
What It Pays And Takes
This is a remote, part-time contractor role for candidates in Germany. The role is suited to people with intermediate experience in AI evaluation or related work.
- Pay: $14 to $36 per hour.
- Location: Open to candidates in Germany.
- Work type: Remote, part-time contract work.
- Language: Fluent in English.
- Bring strong attention to detail, clear written and verbal communication, analytical problem-solving, and sound judgment.
- Be able to manage your time and organize independent remote work.
- Experience with AI model training, RLHF, data labeling, or related evaluation projects is preferred.
- Familiarity with prompt engineering, evaluation rubrics, and rating responses is helpful.
- Experience with machine learning data preparation, large language model projects, or other data-focused AI evaluation work is helpful.