# Eval costs

Eval costs are the arithmetic of judging an agent's traffic. Grading methods differ in per-event
cost by orders of magnitude, and at scale those differences dominate the bill. A deterministic check
costs effectively nothing per event. Grading with a language model pays for inference on every
event, and that price varies widely with the model chosen.

The gap comes from where costs land. Some methods carry their cost per event: every trace a model
judges incurs an inference charge that recurs with traffic. Others carry their cost up front: a
trained classifier's cost is paid once at training time, so grading an additional trace with it
costs close to nothing. Because the per-event methods carry nearly all the marginal cost, total
spend tracks the share of traffic that reaches them multiplied by traffic volume.

This arithmetic decides what evaluation is feasible. Low-probability failures surface only when a
large share of traffic gets examined, so the per-event cost of grading caps how much traffic can be
examined, and with it which failures can be observed at all.

## Questions answered under this concept

- [Does reading every trace cost the same as grading every trace?](https://tessary.ai/answers/eval-costs/does-reading-every-trace-cost-the-same-as-grading)
- [Does running several graders on one trace cost more than running one?](https://tessary.ai/answers/eval-costs/does-running-several-graders-on-one-trace-cost-more)
- [What drives the cost of evaluating an AI agent?](https://tessary.ai/answers/eval-costs/what-drives-the-cost-of-evaluating-an-ai-agent)
- [Which graders belong on every PR and which belong on a nightly run?](https://tessary.ai/answers/eval-costs/which-graders-belong-on-every-pr-and-which-on-nightly)
- [Why do quality layers built on an LLM judge end up sampling instead of grading everything?](https://tessary.ai/answers/eval-costs/why-llm-judge-quality-layers-end-up-sampling)

---

Source: https://tessary.ai/answers/eval-costs
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai