Agent Reliability

The failure hides in the handoff

Multi-agent eval coverage means grading the handoff between agents, not just each agent's own output. Documented failures cluster where one agent's result becomes another agent's input, passing every per-agent grader while the task never completes. When a downstream agent's scores sink, Tessary traces the drop back through the chain to the handoff that shifted, and the change behind it.

Find what broke your agentSee the trace model
The research

More agents rarely raises the score

MAST, a Multi-Agent System Failure Taxonomy built from more than 1,600 annotated traces, found that adding more agents to a system rarely moves the benchmark score the way teams expect. Each agent's own output can look correct on its own grader while the seam where work changes hands goes unchecked. That gap is what the taxonomy's inter-agent misalignment category describes, one of three failure categories it identifies alongside system design issues and task verification.

MAST taxonomy ↗
By the numbers

Where multi-agent failures actually land

MAST's failure shares put the weight on coordination between agents, not on any single agent getting its own task wrong.

36.94%

of annotated failures are inter-agent misalignment: lost context at handoffs, miscommunication, and conflicting outputs

MAST

41.77%

of annotated failures are system design issues, the taxonomy's other large category

MAST

86.7%

task failure rate on benchmark tasks for the worst of the seven frameworks studied; the best still failed 41% of the time

MAST
Two different questions

Did each agent pass, or did the handoff hold?

A per-agent grader confirms one agent's output looks right on its own. None of that confirms what the next agent actually received.

Per-agent grading

What most evals check

  • Confirms each agent's own output matches an expected result
  • Has no visibility into what the next agent actually received
  • Passes even when a handoff drops or corrupts a needed field
Tessary

Handoff-aware coverage

What actually catches it

  • Grades the payload Agent B actually received, not what Agent A said
  • Checks shared state stays consistent across turns, not just within one agent's instructions
  • Flags conflicting outputs the orchestrator picked between without reconciling
How it works

From handoff to root cause in four steps

Trace the handoff, grade what was actually received, and when a downstream score drops, find the change that caused it.

  1. Trace the handoff as its own observation

    A trace model that separates Session, Turn, and typed Observation lets a handoff between two agents show up as something a grader can point at directly, not a step buried inside one agent's log.

  2. Grade what the receiving agent got

    A grader checking Agent B's actual input catches a dropped field that Agent A's own grader would never flag, because it never looks past Agent A's message. The grader doesn't have to be precise, either: Tessary reads the trend across its verdicts over time, so even a rough check on the payload bends visibly when the handoff degrades.

  3. Grade the full trace, not each turn alone

    A task can pass every per-agent check and still fail once you look at whether the end state matches what the user asked for across the whole session.

  4. Trace a drop back to the handoff that caused it

    When a downstream grader's trend drops, Tessary traces the failing quality dimension back through the chain to the handoff that shifted, and names the change behind it: an upstream agent's prompt edit, a tool returning different data, or a model provider update that never touched the downstream repo.
The coordination points

Four coordination points Tessary watches

Graders for each point are drafted from what the handoff is supposed to carry, stored as intent outside any agent's code, and sharpened as drifts are reviewed. Nobody hand-maintains four grader suites per pipeline.

The handoff payload

What Agent A actually passes to Agent B, checked against what Agent B needed, not against whether Agent A's message reads fluently.

Shared state across turns

Facts one agent established that a later agent has to still hold true, distinct from any single agent drifting off its own instructions.

Termination decisions

Which agent decides the task is done, and whether that decision matches what actually happened downstream.

Conflicting outputs

Two agents producing results that can't both be true, where the orchestrator picks one without reconciling the disagreement.
Questions

Answered, plainly.

Multi-agent eval coverage is the practice of writing graders for the coordination points between agents, such as handoffs, shared state, and termination decisions, in addition to grading each agent's own output. A system can pass every individual agent's grader and still fail the task if the handoff between agents drops or corrupts information.
A single-agent grader checks whether one agent's output matches an expected result. It has no visibility into what the next agent in the pipeline actually received. An orchestrating agent can pass along a technically valid but incomplete result, and every per-agent grader involved still reports a pass, because none of them checks the handoff itself.
Because the cause sits upstream. An earlier agent's prompt was edited, its model provider shipped an update, or a tool it calls started returning different data, and its output shifted in a way that still reads as valid. The downstream agent's scores sink three turns later, and nothing in the downstream repo explains it. Tessary reads the trend across grader verdicts to catch the drop, then traces it back through the chain to the upstream shift that fed it.
Inter-agent misalignment is one of three failure categories in MAST, a taxonomy built from more than 1,600 annotated multi-agent traces by researchers including a team at UC Berkeley. It covers failures where agents miscommunicate, lose context during a handoff, or produce conflicting outputs, distinct from a single agent simply getting its own task wrong.
The handoff payload between two agents, shared state that has to stay consistent across turns, the decision to end a task, and conflicting outputs from agents working on the same problem. Each of these can fail silently even when every individual agent's own output looks correct on its own terms.
Sometimes, if the handoff or shared-state check is written as its own grader against the trace. A pre-deploy check can then read an incoming pull request, identify which coordination graders the change touches, and flag the ones at risk of regressing. But that gate only covers changes that ship through a pull request. An upstream agent's shift or a provider model update reintroduces the failure with no deploy at all, which is why the same graders keep watching production.
Keep exploring

Related pages
Get started

Start with the handoff you don't trust

Connect a Langfuse or Braintrust trace, or send OTLP from your own multi-agent pipeline, with no credit card involved. Langfuse will show you that Agent B's score dropped. It won't show that Agent A started omitting a field three turns upstream. That trace-back is what Tessary adds.

Find what broke your agent