AI Personalization Evaluator (Russian)

You will evaluate personalized conversational AI by creating multi-turn prompts based on personal context and reviewing how models use relevant information. You will identify inaccurate personalization, unsupported guesses, hallucinations, forced connections, and unnecessary narration.

You will compare model responses, explain your judgments with clear evidence from the conversation, and check whether summaries and data sources were used correctly. You will also delete evaluation conversations as part of careful data handling.

• Create and run multi-turn prompts that test the use of personal context.
• Check whether responses follow the original prompt and use personalization appropriately.
• Verify that claims about the user are supported by available evidence.
• Compare and rank responses for helpfulness, ease of use, naturalness, and overall quality.
• Write concise rationales that cite specific conversation turns.
• Review debugging information and delete evaluation conversations after review.

This is a remote, part-time contractor engagement lasting three months. The role is open worldwide and requires proficient written and reading ability in Russian.

• Pay: $15 per hour.
• Availability: At least 4 hours per day, up to 40 hours per week.
• Time zone: At least 4 hours of overlap with Pacific Time.
• Workload: 20 or more hours per week.
• Language: Proficient written and reading Russian.
• Skills: Strong analytical reasoning, careful attention to subtle differences, and clear written communication.
• Experience: Ability to create creative multi-turn prompts and recognize inaccurate personalization, unsupported inferences, hallucinations, and forced connections.
• Requirements: A reliable desktop or laptop, internet connection, and willingness to use a primary personal Google account and enable relevant personal data sources.
• Helpful background: A bachelor's degree or equivalent experience in policy, law, ethics, linguistics, journalism, computer science, or a related analytical field.
• Helpful experience: Data annotation, AI quality evaluation, content moderation, or related work.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training work is the human side of building artificial intelligence. Evaluators review model responses and provide evidence-based feedback so AI systems can become more accurate, useful, and natural.