The work
Create and improve datasets used to train and assess large language models. You will use software engineering judgment to review code examples, model-generated solutions, and automated checks.
- Curate Python code examples for backend services, data pipelines, and machine learning infrastructure.
- Write and correct solutions in Python, JavaScript and ReactJS, C or C++, Java, Rust, and Go.
- Assess AI-generated code for architecture, implementation quality, debugging, scalability, and reliability.
- Build Python agents and automated checks that test software solutions and identify recurring errors.
- Evaluate model capabilities across prototyping, architecture and API design, production implementation, launch, experimentation, monitoring, and operational maintenance.
- Work with cross-functional teams to improve AI coding solutions against industry performance benchmarks.
Hours, location, and qualifications
- Location: You must be based in the United States.
- Commitment: 20 or more hours per week.
- Engagement: Individual contractor role, initially one month, with possible extensions based on performance and fit.
- Required experience: At least three years in software engineering, with strong Python skills, including production frameworks and tools.
- You should have experience building full-stack applications and deploying scalable software, plus knowledge of software architecture, design, development, debugging, and code review.
- You must be able to explain technical evaluations clearly in writing and conversation.
- Experience building production systems in high-scale engineering environments and a strong computer science foundation are helpful.