The work
You will evaluate how well large language models handle real software bug-fixing tasks in C# open-source repositories. The work includes reviewing GitHub issues, setting up codebases, checking tests, and running or changing code locally.
- Analyze and triage issues across open-source C# libraries.
- Set up repositories with Docker and local development environments.
- Review unit-test coverage and overall test quality.
- Modify and run real-world codebases to assess LLM bug-fixing performance.
- Work with researchers to identify repositories and issues that create meaningful evaluation challenges.
What it pays and takes
This is a remote, part-time contractor assignment lasting 2 to 3 weeks. The role is available in the listed countries and requires professional fluency in English.
- $15 per approved task.
- 20 or more hours per week.
- Remote work for contractors in Argentina, Bangladesh, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, the Dominican Republic, Ecuador, Egypt, El Salvador, Ghana, Guatemala, Guyana, Haiti, Honduras, India, Mexico, Nicaragua, Nigeria, Pakistan, Panama, Paraguay, Peru, Puerto Rico, Suriname, Turkey, Uruguay, and Venezuela.
- At least 3 years of software engineering experience and strong C# skills.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand complex codebases and run, modify, and test projects locally.
- Experience contributing to or evaluating open-source projects is helpful.
- Previous LLM evaluation or developer-tool experience is a plus.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training is the human work behind systems that learn from examples, including reviewing model responses and testing how models handle code. OpenTrain helps people find these projects, build a profile, and apply for work in the AI training and data-labeling field.