About this role
About Turing Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. You’ll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
About the Role
We are seeking experienced Software Engineers to support a frontier AI training and evaluation initiative focused on improving the coding capabilities of AI agents. In this role, you will review and compare AI-generated coding solutions across real-world software engineering tasks. You will analyze complete agent execution workflows, evaluate code correctness, efficiency, safety, and overall quality, and provide structured ratings and evidence-based feedback to help improve AI-assisted software development.
What does day-to-day look like
- You'll review three AI coding agents that were each given the same real developer request.
- You read each agent's full run: the commands it ran, the files it changed, what it checked and its final message.
- Then you judge which run did the job best.
For every task you
- Rate each run on 10 quality areas, such as following instructions, whether the code works, efficiency, safety, honesty and communication.
- Give each run an overall score from 1 to 5 and rank the three.
- Write a short rationale explaining your ranking, with evidence from the runs.Required Skills: