Data Scientist/Analyst, AI Model Evaluation

You will use Python, data analysis, and clear written reasoning to evaluate AI models and help improve their training. The work combines model testing, response review, dataset development, code review, and analysis of public data.

You will also work with researchers and annotators on supervised fine-tuning, reinforcement learning with human feedback, and reward-model improvement.

• Develop and maintain Python code used to train and optimize AI models.
• Benchmark model performance, analyze evaluation results, and identify useful improvements.
• Evaluate and rank model responses to user questions using set criteria.
• Write clear explanations and rationales for evaluation decisions.
• Create and maintain datasets for supervised fine-tuning.
• Create and refine responses for clarity, relevance, and technical accuracy.
• Review code and documentation, identify issues, and provide constructive feedback.
• Use public datasets, including sources from Kaggle, the United Nations, and the United States government, to answer business questions.

The listing does not provide a pay rate. This is a fully remote, part-time contractor assignment with a one-month contract term.

• Pay: Not provided in the listing.
• Hours: At least 20 hours per week, with options for 20, 30, or 40 hours.
• Schedule: Four hours of daily overlap with Pacific Time.
• Location: Remote and open worldwide.
• Education: A bachelor's or master's degree in engineering, computer science, or equivalent experience.
• Technical skills: Proficiency in Python and strong data analysis ability.
• Reasoning: Strong problem-solving skills and sound business judgment when interpreting data.
• Communication: Ability to explain technical reasoning clearly in Jupyter notebooks or similar formats.
• Language: Fluent conversational and written English.
• Work type: Contractor and part-time.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training work is the human side of building artificial intelligence. People evaluate model responses, prepare datasets, write feedback, and review results so AI systems can improve; experienced contributors are valued for their technical judgment and subject knowledge.