Senior Research Scientist, Agent Evaluation
Skills & Stack
About the Role
Senior Research Scientist, Agent Evaluation at ServiceNow in Hyderabad, building the evaluation layer that validates AI agents before they reach customers: an automated system that scores large volumes of agent traces.
About ServiceNow
Enterprise cloud platform company behind the Now Platform and AI-powered business workflows.
What You'll Do
- 1Design evaluation methodologies and benchmarks for agent reasoning, planning, tool use, reliability and safety across LLM-as-a-Judge, trajectory-based and human evaluation
- 2Take problems from research question to prototype to shipped feature, owning them end to end
- 3Build and harden the pipelines and scoring logic behind customer-facing evaluation
- 4Curate synthetic and real-world datasets
- 5Measure the evaluator itself for consistency and agreement with human labels
What We're Looking For
- 5+ years in ML, applied AI, prompt engineering or agentic AI, including shipping something real to users
- Strong Python skills
- Practical depth in agentic AI and context engineering: planning, reasoning, memory, tool use, retrieval, long-context
- Experience designing evaluation methodologies, not just running evaluations
- Hands-on production work with LLM APIs: prompt engineering, structured output, cost and latency tradeoffs
Check your fit before you apply
We check the resume on your profile against this role at ServiceNow: a match score, what is missing, and what to fix before you apply.
Ready to apply for this role?
Senior Research Scientist, Agent Evaluation at ServiceNow — click below to submit your application.
