Rust LLM Evaluation and Repository Validation Engineer

You will help build training and evaluation datasets for realistic software engineering tasks. The work focuses on Rust repositories, GitHub issues, testing, and assessing how large language models handle bug-fixing workflows.

You will work with researchers to find repositories and issues that present meaningful challenges for language models. The role also calls for senior-level technical judgment, with possible opportunities to lead junior engineers on project work.

• Analyze and triage GitHub issues across open-source libraries.
• Set up repositories, including Dockerization and environment configuration.
• Evaluate unit test coverage and test quality for software tasks.
• Modify and run codebases locally to assess LLM performance on bug-fixing work.
• Work with researchers to identify difficult repositories and issues.
• Provide technical guidance at a senior software engineering level.

The role is a part-time contractor position requiring at least 20 hours per week. The listing does not state a payment rate.

You must be comfortable working in English and be based in one of the listed countries. Strong Rust skills and the ability to navigate complex codebases are central to the work.

• Pay: Rate not listed in the role details.
• Time: 20 or more hours per week.
• Type: Part-time contract work.
• Location: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, or Mexico.
• Language: English.
• Required: Strong Rust experience.
• Required: Experience with Git, Docker, and basic software pipeline setup.
• Required: Ability to run, modify, and test real-world projects locally.
• Preferred: Experience contributing to or evaluating open-source projects.
• Helpful: Prior LLM research or evaluation experience.
• Helpful: Experience with developer tools or automation agents.
• Helpful: Senior software engineering experience in repository analysis and testing.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training work uses human judgment to prepare examples and evaluations that help software systems improve. In this role, your software engineering experience helps assess whether an AI system can understand repositories, fix bugs, and produce reliable code.