Medicine Physician (MD/DO/PhD) Medical AI Evaluator

You will assess how AI systems perform on real medical problems. The work combines clinical judgment, evidence-based medicine, and structured evaluation of model behavior.

The duties and qualifications in this role should be reviewed before applying because the partner title and role description use different role wording.

• Design systematic evaluation frameworks for medical AI systems.
• Create clinical scenarios that test AI reasoning and decision-making.
• Build assessment methods that reflect the complexity of clinical practice.
• Identify gaps in medical knowledge and clinical reasoning.
• Work with AI researchers to improve model performance.
• Use clinical judgment and evidence-based medicine to evaluate AI behavior.

This is a remote, flexible contractor and part-time engagement open worldwide. Scheduling can be arranged around ongoing clinical commitments.

The engagement is described as lasting one month, with possible extensions based on performance and fit. The role calls for 20+ hours per week and allows up to 30 hours per week.

• Pay: Compensation details are not provided.
• Location: Remote and open worldwide.
• Time: 20+ hours per week, with up to 30 hours per week described for the engagement.
• Term: One month, with potential extensions based on performance and fit.
• Experience level: Entry level.
• Required: An active license to practice medicine in any specialty.
• Required: Current clinical practice experience and experience with clinical decision-making.
• Required: A strong foundation in evidence-based medicine.
• Required: Strong analytical and communication skills.
• Helpful: Experience turning complex clinical judgment into clear evaluation criteria, scenarios, or structured assessments.
• Helpful: Familiarity with medical AI, clinical decision support, or evaluating reasoning quality.
• Helpful: Interest in using AI in clinical practice.

Apply on OpenTrain with your resume and then complete the application on the hiring site.

AI training is the human work behind systems that learn from examples. Contributors evaluate model responses, create realistic scenarios, and provide expert feedback so AI systems can handle complex tasks more reliably.

OpenTrain is a platform where people find AI training and data-labeling projects, build a profile, and apply in minutes. Medical expertise is valuable in projects that need careful judgment about clinical information and reasoning.