Healthcare Specialist Rubric Design

You will help train AI systems by creating difficult healthcare questions, writing detailed scoring rubrics, and reviewing AI-generated clinical content. Your work will focus on whether responses are accurate, complete, safe, well reasoned, appropriately uncertain, and professionally relevant.

• Create challenging healthcare questions that reveal weaknesses in advanced AI responses.
• Design rubrics covering essential concepts, reasoning steps, caveats, acceptable alternatives, and critical errors.
• Evaluate clinical content for factual accuracy, completeness, safety, reasoning quality, and appropriate certainty.
• Separate minor omissions from problems that could materially affect clinical quality or safety.
• Review rubrics written by other healthcare professionals.
• Write expert reference answers, explanations, critiques, and gold-standard content for reinforcement learning and supervised fine-tuning workflows.

This is a remote, part-time contractor role for healthcare professionals in the United States. The listed experience level is entry level, and the role requires substantial healthcare knowledge and judgment.

• Pay: Up to $100 per hour.
• Time: 20 or more hours per week.
• Location: Open to candidates in the United States.
• Language: Professional English proficiency at C1 level or higher.
• Qualifications: A professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline.
• Experience: At least three years of professional or clinical experience in a clearly defined healthcare specialty or subspecialty.
• Skills: Create challenging expert-level questions, write detailed rubrics, assess clinical reasoning and safety, judge completeness and uncertainty, and identify important omissions and acceptable alternatives.
• Helpful background: Medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review, quality assurance, AI evaluation, reinforcement learning from human feedback, supervised fine-tuning, data annotation, model benchmarking, or rubric-based scoring.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training is the human work behind systems that learn from examples. Healthcare specialists are needed to judge clinical content carefully, because accurate rubrics and expert feedback help models produce safer and more useful responses.