Software and ML Engineer, LLM Systems Evaluation

The title and description of this role name different fields. Check the duties and qualifications below before you apply.

This contract role focuses on evaluating AI models on advanced LLM inference and systems-engineering problems. You will define what correct, efficient, and well-reasoned technical work looks like rather than deliver production software.

Your work will cover GPU performance, model serving, custom kernels, quantization, and related systems engineering. You will create technical content and assess model responses using your hands-on expertise.

• Design realistic scenarios and problem sets for LLM inference optimization, GPU kernel design, model-serving systems, and systems performance engineering.
• Write reference solutions that explain expert reasoning, implementation tradeoffs, and technical standards.
• Grade model outputs for correctness, performance, and architectural quality.
• Write technical explanations and feedback that distinguish acceptable answers from excellent ones.

This is a remote, part-time contractor role with a flexible schedule.

• Pay: $125 per hour.
• Time: 20 or more hours per week.
• Work arrangement: Remote, part-time contract work.
• Location: Open to candidates in the countries listed for this role.
• Language: Fluent English.
• Education: A bachelor's degree or higher in computer science, computer engineering, electrical engineering, or a related field.
• Experience: Hands-on work with GPU inference optimization, custom CUDA kernel development, model-serving systems, quantization, or optimizer design.
• Professional background: Experience owning systems-engineering work at a recognized technology or AI company.
• Communication: Strong written communication for technical explanations, reference solutions, and model feedback.

The following experience is helpful for this work, but is not listed as required: SGLang, Mamba or Mamba2 architectures, IBM Granite-family models, open-source inference or serving frameworks, distributed inference, continuous batching, and KV-cache optimization.

• SGLang or other open-source inference and serving frameworks.
• Mamba or Mamba2 architectures.
• IBM Granite-family models.
• Distributed inference and continuous batching.
• KV-cache optimization.

Apply on OpenTrain with your resume, then complete the application on the hiring site.

AI training is the human work behind modern AI systems, including writing examples, rating model responses, and checking technical outputs. People with deep specialist knowledge help set reliable standards and explain why one answer is stronger than another.