About this role
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.
Scope of the Role
We're building a dedicated reviewer team to evaluate complex, real-world agentic AI workflows, assessing whether AI agents complete tasks safely, respect user intent and consent, and hold up under close scrutiny. This isn't routine content moderation: reviewers work through ambiguous, multi-step scenarios where judgment calls matter as much as checklists. You'll operate inside isolated test environments, apply structured rubrics, and help calibrate what "safe and policy adherent agent behavior" actually looks like.
What You’ll Own
- Review multi-step agent task trajectories against rubric-based criteria covering task success, safety, and policy adherence
- Evaluate whether an agent respected user agency and informed consent — not just whether it followed literal instructions
- Apply privacy guardrails when reviewing tasks involving sensitive or personal data
- Work inside isolated test environments, including verifying environment resets between test runs to prevent cross-contamination
- Flag and escalate safety-relevant or ambiguous findings through defined escalation paths