Why isn't grading an agent's final answer enough to catch a multi-step failure?

Because a wrong intermediate step can still produce a final reply that reads fine, so grading only the end catches that a session went badly, not which step caused it. AgentEval, a 2026 study of step-level agent evaluation, tested 450 cases across three production workflows and found that 63% of step-level failures were propagated from an earlier step rather than caused locally, with an average propagation chain of 2.1 steps before the error surfaced, and 3.2 steps when the failure involved lost context.

A grader on the final answer sits at the end of that chain, reading the symptom instead of the step that produced it. A grader on each step catches the corrupted one before its output feeds the steps built on top of it, which is also what lets you fix the step that actually broke instead of the one where the failure became visible.

The same shape shows up one level higher: in a multi-agent system, the agent that broke is often not the one that changed.

sources

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y