The work
You will evaluate open-source software repositories and examine how well large language models handle real bug-fixing tasks. The work combines software engineering, test review, environment setup, and hands-on code evaluation.
You will work with public GitHub repositories and collaborate with researchers to find issues and projects that are difficult for AI systems to solve.
- Analyze and triage issues in open-source libraries.
- Configure repositories and development environments, including Docker-based setups.
- Review unit-test coverage and overall test quality.
- Run, modify, and test real-world C# codebases locally.
- Evaluate LLM performance on software bug-fixing tasks.
- Help identify repositories and issues that challenge AI systems.
What it pays and takes
This is a remote contractor assignment open to professionals in Latin America. The contract lasts four weeks, and the work includes four hours of overlap with Pacific Time.
- Pay: $25 per label.
- Time: 20+ hours per week.
- Location: Open to candidates in participating Latin American countries.
- Language: Fluent in English.
- Experience: At least three years of software engineering experience.
- Technical skills: Strong C# or C++ experience, Git, Docker, and basic software pipeline setup.
- You must be able to run, modify, and test real-world projects locally.
- Experience contributing to or evaluating open-source projects is helpful.
- LLM research, evaluation, developer tools, or automation-agent experience is an advantage.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training work uses human judgment to help artificial intelligence systems learn from examples and improve their responses. Software engineers are needed for specialized projects that involve code, testing, and technical evaluation.