The work
You will create and assess software engineering examples used to train and benchmark large language models. The work combines hands-on coding with structured technical evaluation.
- Curate code examples and create precise solutions to software engineering tasks.
- Correct and refine implementations in Python, C/C++, Rust, Go, Java, and JavaScript, including ReactJS.
- Review AI-generated code for correctness, efficiency, scalability, and reliability.
- Build agents and automatic checks to verify infrastructure and systems code, and identify recurring errors.
- Assess model capabilities in prototyping, architecture and API design, production implementation, monitoring, and maintenance.
- Work with cross-functional teams to improve AI coding solutions against performance benchmarks.
Schedule and qualifications
- Location: Open to candidates in the United States.
- Commitment: 20+ hours per week; the schedule is flexible.
- Engagement: Part-time contractor role with an initial one-month duration and possible extensions based on performance and fit.
- Skills: Strong systems programming, infrastructure, or backend development expertise, and experience building and deploying scalable production software.
- Knowledge: Software architecture, design, development, debugging, and code-quality assessment.
- Communication: Ability to give clear, structured written and verbal explanations of technical evaluations.
- Helpful background: A strong computer science foundation, experience shipping products in a fast-moving engineering environment, or work with performance-critical applications, infrastructure systems, production services, code review, automated verification, or engineering workflow evaluation.
- Language: Fluent in English.