ML Evaluation Data Analyst

As an ML Evaluation Data Analyst, you will study data from machine learning training, inference, and evaluation workflows. You will assess model behavior, investigate failures, and create reliable analysis that helps improve AI systems.

The role combines data analysis, metric validation, data quality checks, and clear reporting. You will work with ML engineers and researchers on difficult real-world evaluation tasks.

• Analyze structured and unstructured data from machine learning workflows.
• Define, calculate, and validate metrics used to assess model performance and behavior.
• Investigate data distributions, edge cases, model outputs, and benchmark failures.
• Use Python and SQL to analyze data and support evaluation workflows.
• Check datasets and experiments for quality, consistency, and correctness.
• Create clear reports, analytical materials, and reproducible analysis workflows.

This is a part-time, remote contractor assignment for an intermediate-level analyst. The contract is planned for three months and may be adjusted based on engagement.

• Time requirement: At least four hours per day and 20 hours per week.
• Schedule: At least four hours of overlap with Pacific Standard Time.
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Brazil, and Mexico.
• Language: Strong spoken and written English.
• Experience: At least three years as a data analyst or analytics-focused engineer.
• Technical skills: Strong Python for data analysis and solid SQL with relational datasets.
• Knowledge: Experience with machine learning outputs and evaluation metrics.
• Strengths: Statistical understanding, analytical reasoning, and experience with large, complex datasets.
• Work quality: Clean, readable, well-documented analytical code and clear communication.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training work is the human work behind modern AI systems, including reviewing model outputs, checking data, and measuring how well models perform. Analysts with strong technical and statistical skills are paid to make these evaluations accurate, consistent, and useful.