The work
You will help create training and benchmark datasets that assess how well AI models handle software engineering. The work involves reviewing and refining code, writing precise solutions, and explaining your judgments clearly.
- Create or correct code examples in Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go.
- Review AI-generated code for efficiency, scalability, reliability, and performance.
- Build verification agents and other checks that find recurring errors and validate solutions automatically.
- Assess model abilities across prototyping, architecture and API design, production implementation, launch, experimentation, monitoring, and ongoing maintenance.
- Work with cross-functional teams to improve AI coding solutions.
What you need
- Strong full-stack application development skills and production deployment experience.
- Deep knowledge of software architecture, design, development, debugging, and code review.
- Working ability in multiple programming languages named above.
- Fluent in English, with strong spoken and written communication skills for clear, structured evaluation explanations.
- Experience designing automated verification methods and reasoning across the software development lifecycle is helpful.
Schedule and location
- Contractor engagement with flexible scheduling and a requirement of 20 or more hours per week.
- Open to candidates based in the United States.
- The initial engagement is one month, with possible extensions based on performance and fit.