LLM Java Developer: AI Model Evaluation

You will combine Java development with hands-on AI training work. Your work will support large language model training, evaluation, and improvement through code, datasets, response reviews, and written reasoning.

• Design, develop, and maintain efficient Java and backend code for AI model training and optimization.
• Build modular web applications with scalable architectures and review code and documentation for quality.
• Evaluate and rank model responses to user questions using defined quality criteria.
• Write clear explanations and rationales that show sound reasoning and technical judgment.
• Create and maintain datasets for supervised fine-tuning, where examples teach a model how to respond.
• Work with researchers and annotators on reinforcement learning from human feedback and reward-model improvement.
• Develop evaluation strategies that support user needs and ethical model behavior.
• Refine model responses and assess technical accuracy, clarity, relevance, and alignment with standards.
• Analyze evaluation results to benchmark model performance and support continuous improvement.

This is an entry-level, remote contractor role with part-time availability. The listing does not provide a pay rate.

• Hours: 20 or more hours per week.
• Location: Open worldwide; the work is remote.
• Languages: Professional fluency in English for spoken and written communication.
• Experience level: Entry level.
• Programming: Strong knowledge of Java syntax, conventions, and modern coding practices.
• Code quality: Ability to produce clean, organized, correct, clearly annotated code with attention to readability, security, stability, and testing.
• Evaluation: Ability to assess technical responses against explicit criteria and explain decisions clearly.
• Education: A bachelor's or master's degree in engineering or computer science, or equivalent experience.
• Helpful background: Software quality assurance and test planning experience; experience with large language models, supervised fine-tuning, reinforcement learning from human feedback, or model evaluation.

Apply on OpenTrain with your resume, then complete the application on the hiring site. OpenTrain is where you find this role and begin the application; it does not employ or pay contractors for the work.

AI training is the human work behind systems that learn from examples. Contributors write, review, rank, and improve model responses, code, and datasets so AI systems become more useful, accurate, and consistent; OpenTrain helps people find these roles and build a record of their AI training experience.