The work
Create and review coding examples used to train and evaluate large language models. You will assess AI-generated code across software development tasks, from early designs to production and ongoing operations.
- Create code examples, write solutions, and correct code for Python backend and machine learning workflows.
- Develop and evaluate JavaScript solutions across React, Node.js, frontend, and API work.
- Review additional solutions in C/C++, Java, Rust, and Go.
- Judge code for architecture, debugging, efficiency, scalability, reliability, and production quality.
- Build agents that check code quality, find recurring errors, and test software engineering solutions.
- Assess model capabilities in prototyping, architecture and API design, production implementation, monitoring, and operational maintenance.
What it pays and takes
- Hours: 20 or more per week.
- Location: Remote, open to candidates in the United States.
- Engagement: Part-time contractor role, initially for one month, with possible extensions based on performance and fit.
- At least three years of software engineering experience.
- Strong full-stack development skills in Python and JavaScript, including React and Node.js.
- Experience deploying scalable, production-grade software using modern languages and tools.
- Strong knowledge of software architecture, design, development, debugging, and code review.
- Ability to explain evaluation decisions clearly in writing and conversation.
- Experience with intelligent systems or advanced AI software is helpful. Familiarity with C/C++, Java, Rust, Go, automated verification, agents, or model benchmarking is also useful.
- Fluent in English.