Data Science & Analytic Methodology SME

You will help evaluate how large language models perform in real-world data science and analytic workflows. The work includes creating structured evaluation cases, reviewing model responses, and explaining where automated analysis succeeds or fails.

• Map scaled-analysis and pipeline-integration scenarios to an established evaluation taxonomy.
• Create detailed cases covering automated triage, data enrichment, analysis at scale, and realistic failure modes.
• Evaluate model responses for source handling, reasoning, stated limitations, and practical usefulness.
• Identify over-refusal that could block legitimate analytic work or interrupt pipeline continuity.
• Document difficult edge cases and emerging concerns in analytic methodology.
• Give clear written and verbal recommendations to support ongoing improvement.

This is a remote, part-time contractor role. The listed experience level is entry level, while the work calls for practical experience with large language models, large-scale analysis, and data or analytic pipelines.

• Pay: $60 to $100 per hour.
• Work arrangement: Remote, part-time contract work.
• Location: Open to candidates in the countries listed for this role, including the United States, Canada, the United Kingdom, India, Japan, and other supported countries.
• Language: Fluent written English and the ability to explain complex technical ideas clearly.
• Experience integrating large language models, including government GPT variants or equivalent systems, into complex data or analytic pipelines.
• Experience conducting analysis at scale rather than only one-off analysis.
• Understanding of automation failure modes, evaluation metrics, and analytic methodology.
• Experience teaching, mentoring, or providing technical leadership in data science or a related discipline.
• Familiarity with evaluation frameworks or benchmark design.
• Helpful background: defense, intelligence, or government analytic experience, plus experience working in distributed international settings with minimal clearance constraints.

You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.

AI training is the human work behind systems that learn from examples. Specialists review model outputs, test real-world behavior, and give detailed feedback so AI systems become more useful and reliable.