The work
You will help evaluate large language models by working with public GitHub repositories and real C# software issues. The work includes preparing codebases, reviewing tests, and assessing how well models handle bug-fixing tasks.
You will also work with researchers to find repositories and issues that create useful challenges for LLM evaluation.
- Analyze and triage issues in open-source C# libraries.
- Set up repositories with Docker and local development environments.
- Review unit-test coverage and overall test quality.
- Modify, run, and test real-world codebases locally.
- Evaluate LLM performance on software bug-fixing tasks.
- Help identify repositories and issues suitable for meaningful evaluation.
What it pays and takes
This is a remote contractor assignment for 2 to 3 weeks. The role requires practical software engineering experience and the ability to work independently with complex codebases.
- Pay: $25 per approved task.
- Commitment: At least 4 hours per day and 40 hours per week.
- Schedule: Include at least 4 hours of overlap with Pacific Time.
- Location: Open to candidates in Argentina, Bangladesh, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, the Dominican Republic, Ecuador, Egypt, El Salvador, Ghana, Guatemala, Guyana, Haiti, Honduras, India, Mexico, Nicaragua, Nigeria, Pakistan, Panama, Paraguay, Peru, Puerto Rico, Suriname, Turkey, Uruguay, Venezuela, and other eligible Latin American countries.
- Language: Fluent in English.
- Experience: At least 3 years of software engineering experience.
- Technical skills: Strong C# experience, Git, Docker, and basic software pipeline setup.
- You must be able to understand complex codebases and run, modify, and test projects locally.
- Helpful experience includes contributing to or evaluating open-source projects.
- LLM evaluation or developer-tool experience is a plus.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training uses human review and examples to improve how artificial intelligence systems work. In this role, your software engineering judgment helps assess whether language models can solve real coding problems accurately.