Opportunity at
ServiceNowSenior Research Scientist, Agent Evaluation
This full-time, on-site Senior Research Scientist role focuses on agent evaluation at ServiceNow in Hyderabad. The scientist owns parts of an automated evaluation platform that scores…
Career Tools
About This Role
This full-time, on-site Senior Research Scientist role focuses on agent evaluation at ServiceNow in Hyderabad. The scientist owns parts of an automated evaluation platform that scores large volumes of agent traces, including the pipeline that runs traces through model-based judges and the scoring logic that turns output into actionable results. The job covers designing evaluation methods for reasoning, planning, tool use, reliability and safety, and turning research questions into shipped…
Job Description
This full-time, on-site Senior Research Scientist role focuses on agent evaluation at ServiceNow in Hyderabad. The scientist owns parts of an automated evaluation platform that scores large volumes of agent traces, including the pipeline that runs traces through model-based judges and the scoring logic that turns output into actionable results.
The job covers designing evaluation methods for reasoning, planning, tool use, reliability and safety, and turning research questions into shipped features. It suits an applied AI researcher with five or more years of experience.
Responsibilities
- Design evaluation methodologies and benchmarks for agent reasoning, planning, tool use, reliability and safety
- Take research questions through prototypes to shipped features, owning them end to end
- Build and harden the pipelines and scoring logic behind customer-facing evaluation
- Curate synthetic and real-world datasets and measure how consistent the evaluators are
Requirements
- Five or more years in machine learning, applied AI, prompt engineering or agentic AI, including shipping to users
- Strong Python skills
- Practical production work with LLM APIs, covering structured output and cost and latency trade-offs
Qualifications
- Experience designing evaluation methods, not just running evaluations
- Practical knowledge of planning, memory, tool use, retrieval and long-context work in agentic AI
- Experience with AI-assisted development tools such as Claude Code or Windsurf is a plus
Core Skills
Frequently Asked Questions
Answers are based only on the employer’s listing; where it doesn’t say, neither do we.
Where is the job?
The listing places the role in Hyderabad, India, as on-site work.
What experience is needed?
The posting asks for five or more years in machine learning, applied AI or agentic AI, including shipping work to users.
Which programming language is required?
Strong Python skills are required.
Sample Interview Questions
AI-generated questions tailored to this specific role — a preview of the full practice set.