AI Model Evaluation Data Scientist

You will develop Python solutions and analyze datasets that help improve AI models. The work combines model evaluation, response ranking, supervised fine-tuning, and reinforcement learning with human feedback.

You will also explain your decisions clearly, review code and documentation, and turn complex findings into practical improvements for AI systems.

• Write and maintain Python code for training and optimizing AI models.
• Benchmark model performance and analyze evaluation results.
• Evaluate and rank model responses to user queries using set criteria.
• Write clear explanations and rationales for evaluation decisions.
• Create and maintain datasets for supervised fine-tuning.
• Help researchers and annotators improve reinforcement learning with human feedback and reward models.
• Create and refine responses for clarity, relevance, and technical accuracy.
• Review code and documentation, identify issues, and give constructive feedback.
• Use public datasets, including Kaggle, United Nations, and United States government data, to answer business questions.

This is a fully remote contractor assignment with a one-month contract term. The role is listed as entry level and requires technical data analysis, clear written reasoning, and effective communication with researchers and other stakeholders.

• Expected commitment of at least 20 hours per week.
• Choose 20, 30, or 40 hours per week.
• Four hours of daily overlap with Pacific Time is required.
• Open worldwide as a remote assignment.
• Fluent conversational and written English.
• Proficiency in Python and strong data analysis skills.
• Strong problem-solving ability and sound business judgment when interpreting data.
• Ability to explain technical reasoning clearly in Jupyter notebooks or comparable formats.
• Bachelor's or master's degree in engineering or computer science, or equivalent experience.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training is the human work behind modern AI systems, including reviewing model responses, preparing datasets, evaluating results, and giving feedback. People with technical experience help models improve by applying careful analysis and clear judgment to challenging examples.