About this role
Engagement Type: Business-to-Business (B2B) / Independent Contractor
Location: Remote
The Opportunity: Define the "Ground Truth" for Generative AI
We are looking to engage elite Subject Matter Experts (SMEs) to partner with us on high-impact AI evaluation and training projects. We are not looking for generalists; we are seeking established authorities—the top 1% in their respective fields—who are eager to apply their deep domain knowledge to shape the reasoning, accuracy, and safety of next-generation Large Language Models (LLMs).
This is a consulting engagement for independent professionals who want to help define what "perfection" looks like in their industry. You will not just be grading AI; you will be architecting the "Gold Standard" rubrics that teach models how to think like an expert.
Scope of Services & Deliverables
This engagement focuses on high-level advisory and specialized analysis. Consultants will operate under a Statement of Work (SOW) to provide the following deliverables:
- Rubric Architecture: Review and advise on the quality criteria used to evaluate AI outputs. You will determine if the AI’s logic holds up to professional scrutiny in your field.
- "Golden" Data Curation: Create, edit, or certify "Golden Tasks"—benchmark examples of perfect prompts and responses that serve as the ground truth for model training.
- Expert Gap Analysis: Analyze model failures (hallucinations, logical fallacies) and provide technical breakdowns of why the model failed and how a human expert would have approached the problem.
- Advisory on Complexity: Consult on the difficulty and nuance of specific prompts to ensure the AI is being tested against sufficiently rigorous scenarios.