How do you evaluate an AI agent?
Build a set of realistic tasks, run the agent on each several times, and grade the outcome — with code checks where the answer can be verified, a model-based grader where judgement is needed, and human review to calibrate both. Anthropic suggests that 20 to 50 simple tasks drawn from real failures is a good start. Track capability (can it do new things?) separately from regressions (does it still do what it did?), and measure consistency, not just a single success.
- Published
- 22 October 2026
- Sources checked
- 22 October 2026
- From
- Kramiva
01
The parts of an agent eval
Terms from Anthropic's “Demystifying evals for AI agents”.
- Task: one input with a defined success condition
- Trial: one run of the agent on a task; agents vary, so run several
- Grader: the logic that scores a trial
- Transcript: the full record of what the agent did
- Outcome: the end state the grader judges — for example, whether a record was actually updated
- Harness: the code that runs tasks and collects results
02
Choosing graders
Code-based graders are fast, cheap and objective: did the test pass, does the record exist, is the number right? Model-based graders handle open-ended quality, but need checking against human judgement. Human graders are the most reliable and the slowest, so they are best used to calibrate the other two.
Grade the end state where you can. Anthropic's research team made the same point: judging whether the agent reached the right outcome is more robust than checking that it followed a particular path.
03
Measuring consistency
One successful run proves little. pass@k asks whether at least one of k attempts succeeded — useful when a person picks the best output. pass^k asks whether all k succeeded — the right measure when users rely on the agent working every time.
04
Trade-offs
Evals take time to build and need maintaining as the product changes, and a model grader can be wrong in systematic ways. Without them, though, every prompt change or model upgrade is a guess. Starting small with real failures keeps the effort proportionate.
Where Kramiva fits
Both Kramiva AI services include evaluation: criteria agreed before the build and measured after it, and for in-product features a task-specific evaluation set re-run on every change.
Sources
- Demystifying evals for AI agents — Anthropic, 9 January 2026
- How we built our multi-agent research system — Anthropic, 13 June 2025
Accessed 22 October 2026. Kramiva is not affiliated with these publishers.
Related questions
Tell us what you are building.
A short brief gets you a real reply from a founder within one business day — an honest read on fit, scope, and budget.