The work
You will review AI-generated Python code in real open-source projects and judge its quality, performance, and ability to fix bugs. The work includes running code locally, checking tests, and helping researchers find difficult tasks for AI models.
- Analyze and triage GitHub issues in active open-source libraries.
- Set up and configure repositories, including Docker environments.
- Review unit test coverage and assess test quality.
- Modify and run codebases locally to evaluate AI performance on bug-fixing tasks.
- Work with researchers to identify repositories and issues that challenge AI coding systems.
What it pays and takes
This is a remote contractor role with part-time classification and a short project period.
- Pay: $20 per approved task.
- Time: At least 20 hours per week, with four hours of overlap with Pacific Time.
- Location: Open to candidates in India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Türkiye, or Mexico.
- Language: Professional fluency in English.
- Contract length: Two to three weeks, with an expected start next week.
- Strong Python experience.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfort running, modifying, and testing real-world projects locally.
- Experience with open-source projects, LLM research or evaluation, developer tools, or automation agents is helpful.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.