Agent Reliability

Grade every step, not just the reply

Grading only an agent's final reply misses errors introduced mid-chain. Tessary runs a grader at each step in an agent's own sequence of tool calls and state updates, catching a corrupted step before it feeds a wrong answer into the steps after it.

Find what broke your agentSee the evals SDK
The math

The arithmetic doesn't forgive it

A chain of steps each individually correct 95% of the time completes end to end far less often than intuition suggests.

95%

per-step accuracy that feels safe in isolation

Medium analysis by Flavius Dinu, June 2026

59%

end-to-end completion rate across 10 chained steps at that accuracy

Medium analysis by Flavius Dinu, June 2026

7%

end-to-end completion rate across 50 chained steps at that accuracy

Medium analysis by Flavius Dinu, June 2026
The research

Most failures started somewhere earlier

AgentEval, a 2026 ACL Industry Track paper on step-level agent evaluation, tested 450 cases from three production workflows and found that 63% of step-level failures were propagated from an earlier step rather than caused locally, with an average propagation chain of 2.1 steps before the error surfaced (3.2 steps for failures involving lost context). The step where a failure becomes visible is usually not the step where it started.

Read the AgentEval paper ↗

63%

of step-level failures propagated from an earlier step

2.1

steps in the average propagation chain before a failure surfaced
Two approaches

Whole-chain checks vs step-level checks

Task and session checks aren't wrong. They grade the wrong unit for a single agent's own multi-step chain.

Final-reply / session checks

Grade the chain as one unit

  • Reads only the agent's last message or the session as a whole
  • Can't say which internal step introduced a wrong result
  • A corrupted step two rides along undetected until step five inherits it
  • No visibility into planning steps, tool calls, or state updates individually
Tessary

Step-level grading

Grades each call site on its own terms

  • Checks every planning step, tool call, and state update individually
  • Flags the exact step where a result went wrong
  • Traces upstream, one link at a time, to the step that actually changed
  • Catches drift even when no code change explains it
How it works

Five steps from trace to named step

Tessary ingests your existing trace source, drafts graders per call site, and names which step broke when the trend shifts.

  1. Connect what you're already sending

    Pull a slice from an existing Langfuse or Braintrust source, or send OTLP directly, with no re-instrumentation required to see the first result.

  2. Let Tessary draft the first graders

    Each call site in the chain gets a grader drafted from how your agent actually behaves in production, not a hand-written rubric per step.

  3. Correct the drafts that miss

    The owner of a given step reviews the disagreements that come up and sharpens the grader from there.

  4. Watch the trend, not one call

    Verdicts across each step accumulate into a rate over time, so a real shift stands out from an imperfect grader's normal background noise.

  5. Get the step and the cause

    When a step's rate breaks, the alert names the step and traces back through the chain to the change behind it, in time to act on it.
What you get

What step-level grading catches

Four failure modes a final-reply or session check misses because it grades the chain as one unit instead of a sequence of steps.

The corrupted intermediate step

A tool call that returns a subtly wrong value gets flagged where it happens, not three steps later when the final answer is already wrong.

A no-diff cause

When a step degrades because an upstream agent's output shifted or a tool started returning different data, the trace back through the chain still finds it, even with nothing to point to in a pull request.

The step, not just the session

Step-level grading carries the same call-site idea across a multi-step sequence, so you know which call in the chain started to slip, not just that the session came up short.

A grader that keeps watching

Once a step's grader exists, it runs on every later deploy, so the same step doesn't quietly break twice.
FAQ

Questions about step-level grading

Multi-step agent evals are checks that run against each step in an AI agent's own chain of tool calls and intermediate state, not just its final reply. A single agent working through a task, planning, calling a tool, updating state, calling another tool, can produce a wrong intermediate result that a check on the final message alone won't catch, because later steps build on it and the final output can still look plausible.
A final-output check reads the last message an agent sends and grades whether it looks right. Step-level grading reads each intermediate call in the chain, planning, tool calls, state updates, and grades whether that specific step was correct against what the next step needed from it. An agent can produce a fluent final answer while an early step already fed it the wrong input.
A session-level check grades whether a whole conversation reached its goal. That still treats everything between the user's opening message and the agent's last reply as one unit. A single agent can run ten internal steps inside what looks like one turn of that conversation, and a session grader has no way to say which of those ten steps introduced the problem, only that the session overall came up short.
Yes. A step can start failing because a tool it calls returns differently shaped data, because an upstream agent feeding it changed its output, or because the model provider updated the model underneath it, none of which show up in a repository diff. Tessary reads the trend across a step's grader verdicts over time, so a shift like that still surfaces as a real change even though nothing shipped through a pull request.
No. Tessary drafts an initial grader for a given call site from the agent's own observed production behavior rather than requiring a hand-labeled set up front. The owner of that step corrects the draft against real disagreements, and the grader sharpens from there as more of the chain gets reviewed.
Keep exploring

Related pages
Get started

Grade the step, not just the outcome.

Connect one multi-step flow's existing trace source and see which of its internal steps Tessary would flag first, no credit card required.

Find what broke your agentNo re-instrumentation required if your agent is already on Langfuse or Braintrust.