The work
Create and assess software engineering datasets used to train and benchmark large language models. The work centers on Python and covers backend services, data pipelines, machine learning infrastructure, full-stack development, and related programming languages.
- Curate code examples and write accurate reference solutions.
- Correct AI-generated code for efficiency, scalability, reliability, and quality.
- Build Python agents and automated tools to find error patterns and check solutions.
- Assess models on prototyping, architecture and API design, production implementation, monitoring, and maintenance.
- Create ways to verify software engineering tasks and compare results with benchmarks.
- Work with researchers and cross-functional teams on AI coding projects.
What it pays and takes
- Pay: Not provided.
- Time: At least 20 hours per week.
- Location: Remote work for candidates based in the United States.
- Engagement: Individual contractor role, initially one month, with possible extensions based on performance and fit.
- Required skills: Strong Python skills, including production frameworks, tools, and best practices; experience building full-stack applications and deploying scalable software; and knowledge of software architecture, development, debugging, and code review.
- You should be able to write clear, structured explanations for your evaluations.
- Knowledge of JavaScript, ReactJS, C or C++, Java, Rust, or Go is helpful.
- Experience with high-scale production systems and a strong computer science foundation are helpful. Practical engineering ability is valued alongside formal education.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.