AI Model Evaluation Data Scientist/Analyst

You will help improve AI systems through model evaluation, data analysis, coding, and written reasoning. The work combines technical analysis with clear explanations of why a model response meets or misses set criteria.

You will work with researchers and annotators on supervised fine-tuning, reinforcement learning with human feedback, and reward-model refinement. You will also use public datasets to answer business questions and turn findings into useful improvements.

• Design, develop, and maintain Python code for training and optimizing AI models.
• Benchmark model performance and analyze evaluation results.
• Evaluate and rank model responses to user queries against predefined criteria.
• Write clear explanations and rationales for evaluation decisions.
• Create and maintain task-specific datasets for supervised fine-tuning.
• Create and refine responses for clarity, relevance, and technical accuracy.
• Review code and documentation, identify issues, and give constructive feedback.
• Use public datasets from sources such as Kaggle, the United Nations, and the US government.

This role is suited to a data scientist or analyst who can solve problems, interpret data with sound business judgment, and explain technical reasoning clearly. You should be comfortable communicating with researchers and other stakeholders.

• Proficiency in Python and strong data analysis skills.
• A bachelor's or master's degree in engineering, computer science, or equivalent experience.
• Clear technical explanations in Jupyter notebooks or comparable formats.
• Strong problem-solving skills and sound judgment when interpreting data.
• Effective written and conversational communication in English.
• Remote contractor assignment with a one-month contract term.
• At least 20 hours per week, with options for 20, 30, or 40 hours.
• Four hours of daily overlap with Pacific Time.
• Open worldwide.

Apply on OpenTrain with your resume, then complete the application on the hiring site. OpenTrain is a free platform where people find AI training and data-labeling projects, build a profile, and apply to roles across the industry.

AI training is the human work behind modern AI systems, including reviewing model outputs, preparing datasets, writing responses, and rating results. People with data science and technical skills help make evaluations more accurate and turn complex findings into better model behavior.