# Graders

A grader is a check that reads a trace of an agent's behavior and returns a verdict about one
specific thing the agent was supposed to do. Running one grader across many traces turns a general
sense of quality into a rate for one behavior, and a rate can be compared across agent versions and
over time.

Graders come in three forms, ordered by cost per verdict. Deterministic checks are code that
inspects the trace, cheap enough to run on every output. Trained classifiers read text and score a
single property at low cost. LLM judges use a model to reason about the output. They cost the most
per verdict and cover judgments that require reasoning about meaning.

Two properties determine what a grader's verdict means. The first is scope. A grader tied to one
observable behavior, with a stated condition for when it applies, gives a verdict that says exactly
which behavior failed. As scope widens toward overall quality, each verdict carries less information
about what actually happened. The second is origin. A grader written from a failure observed in real
traffic measures something that actually occurs. A grader written from speculation measures what its
author imagined, which may differ from what the agent does in practice.

Graders also get stale when the agent's intent changes. Each grader encodes what the agent was
supposed to do when it was written, so when the intent changes, the grader keeps judging against the
old one. It still returns verdicts, so its rate can move for reasons unrelated to what the agent is
now meant to do.

## Questions answered under this concept

- [Does a regression grader have to be an LLM judge?](https://tessary.ai/answers/graders/does-a-regression-grader-have-to-be-an-llm-judge)
- [What stops a description from producing a grader that measures the wrong thing?](https://tessary.ai/answers/graders/drafted-grader-measuring-wrong-thing)
- [How do you know when a grader no longer matches what the agent does?](https://tessary.ai/answers/graders/grader-no-longer-matches-the-agent)
- [What does an eval score actually claim about my agent?](https://tessary.ai/answers/graders/what-an-eval-score-actually-claims)
- [What is a grader?](https://tessary.ai/answers/graders/what-is-a-grader)

---

Source: https://tessary.ai/answers/graders
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai