ServiceNow logo
fulltime·ServiceNow

Senior Research Scientist, Agent Evaluation

Hyderabad, IndiaSalary not disclosed6 skills

Skills & Stack

Agent evaluationLLM-as-a-JudgePythonAgentic AIPrompt engineeringBenchmarking

About the Role

Senior Research Scientist, Agent Evaluation at ServiceNow in Hyderabad, building the evaluation layer that validates AI agents before they reach customers: an automated system that scores large volumes of agent traces.

About ServiceNow

Enterprise cloud platform company behind the Now Platform and AI-powered business workflows.

What You'll Do

5 items
  • 1Design evaluation methodologies and benchmarks for agent reasoning, planning, tool use, reliability and safety across LLM-as-a-Judge, trajectory-based and human evaluation
  • 2Take problems from research question to prototype to shipped feature, owning them end to end
  • 3Build and harden the pipelines and scoring logic behind customer-facing evaluation
  • 4Curate synthetic and real-world datasets
  • 5Measure the evaluator itself for consistency and agreement with human labels

What We're Looking For

  • 5+ years in ML, applied AI, prompt engineering or agentic AI, including shipping something real to users
  • Strong Python skills
  • Practical depth in agentic AI and context engineering: planning, reasoning, memory, tool use, retrieval, long-context
  • Experience designing evaluation methodologies, not just running evaluations
  • Hands-on production work with LLM APIs: prompt engineering, structured output, cost and latency tradeoffs

Check your fit before you apply

We check the resume on your profile against this role at ServiceNow: a match score, what is missing, and what to fix before you apply.

Ready to apply for this role?

Senior Research Scientist, Agent Evaluation at ServiceNow — click below to submit your application.

Apply Now

Get new tech jobs by email

Only new listings. No spam. Or get them on Telegram

Prepare for this interview

Free practice for the problem-solving rounds most tech roles screen on.

blur