Agent Reliability

Grade the turn, not the transcript

A transcript-based eval can score every one of a caller's calls as correct: the text of the reply is accurate, and there's no line in a passing report for a caller who got talked over. Voice agent evals have to grade the turn itself.

Find what broke your agentSee multi-turn agent evals
The problem

Interruptions aren't an edge case

A 2026 survey of more than 450 voice agent builders found interruption handling sitting alongside speech-to-text accuracy as a core requirement, not a nice-to-have.

47.5%

of builders say their voice agents interrupt callers mid-sentence

AssemblyAI, 2026 Voice Agent Report

74%

call conversational intelligence a non-negotiable requirement

AssemblyAI, 2026 Voice Agent Report

76%

rank speech-to-text accuracy as non-negotiable, right alongside it

AssemblyAI, 2026 Voice Agent Report
The signal

One suite, opposite backchannel behavior

IHBench, a 2026 benchmark built specifically to grade what happens after a voice agent gets interrupted, tested 27 audio-model configurations on backchannel handling: a caller saying “mm-hm” or “right” without meaning to take the floor. GPT-family models resumed the interrupted sentence correctly only 7% to 31% of the time, while Gemini 2.5 models did closer to 62% to 68%. Two agents built on the same transcript-only test suite can pass identically and still handle a real caller's “uh-huh” in opposite ways, and nothing in a text-based eval report would show the gap.
Two failure surfaces

Transcript grading vs turn-level grading

A transcript grader checks whether the words in a response were the right words. It has no way to check whether those words arrived on time, or whether the agent handled being cut off gracefully.

Transcript-only eval

Grades the reply after the turn is already over

  • Checks whether the final words were correct
  • Never sees how fast the agent yielded when talked over
  • Can’t tell a real interruption from a backchannel like “mm-hm”
  • Misses a dropped correction the caller made mid-sentence
Tessary

Turn-level eval

Grades the audio turn itself

  • Grades how fast the agent yields the floor when talked over
  • Distinguishes a backchannel from a real interruption
  • Checks whether the agent recovers at the right point afterward
  • Reads production call traces, not a fixed test transcript
How it works

How Tessary grades interruption handling

Tessary reads production call traces at the turn level, then reads the trend across those verdicts to separate a real shift from an imperfect grader's usual noise.

  1. Read production call traces at the turn level

    Tessary reads how long the agent took to yield when talked over, whether a backchannel got treated as a full interruption, and whether the agent picked the conversation back up at the right point afterward or dropped what the caller said next.

  2. Draft a grader for each failure surface

    Each of those becomes its own grader, drafted from what your agent actually does on real calls and corrected by the person who owns what it’s supposed to do, rather than written by hand from a rubric.

  3. Carry turn structure through the same trace model

    The same trace model that grades multi-turn conversations carries the call’s turn structure too, so nothing about who was talking when gets flattened before grading runs.

  4. Read the trend across verdicts, not one call

    A grader that’s imperfect still misfires at roughly the same rate call after call, so a real jump in backchannel-handling failures stands out from that steady background noise.

  5. Trace a broken trend to what changed upstream

    When the trend drops, Tessary checks what changed on the voice agent: an ASR vendor changing how it segments pauses, a telephony provider’s jitter climbing, or the model behind the voice pipeline getting swapped in behind the same API name.
What you get

What turn-level grading catches

None of these failures live in the words of the final response, so a suite built only to grade text never sees them.

Backchannel mishandling before it reaches more callers

A grader built for the interruption itself catches an agent that stops mid-sentence for a "yeah, go on," instead of waiting for the pattern to show up as a spike in dropped calls.

A recovery-rate drop, pinned to what changed

A prompt edit, an ASR vendor update, or a model swapped in underneath you, none of which needs a pull request to reach production.

A trend across live calls, not one test transcript

The same graders run continuously on production traffic, so a shift shows up in this week’s calls instead of next quarter’s test suite run.
FAQ

Questions about voice agent evals

Voice agent evals grade a voice agent’s audio turns, not just the text of its replies: how quickly it yields when a caller talks over it, whether it treats a short backchannel like "mm-hm" as a real interruption, and whether it picks the conversation back up correctly afterward. A transcript-only eval only checks the words in the final response, so an agent can pass every transcript test and still frustrate real callers who get talked over or cut off mid-sentence.
Most eval suites built for text agents grade the reply after the audio turn is already over, so they never see how the turn itself played out: the pause before the agent yielded, the backchannel it misread as a full interruption, or the correction it dropped instead of acting on. Those failures live in the turn-taking behavior, not in the words of the final response, so a suite built only to check text keeps reporting green while callers keep getting talked over.
Turn-taking latency is how quickly the agent yields the floor once a caller starts speaking; most production systems now do this in well under a second. Interruption recovery is a separate question: once the agent stops, does it pick the conversation back up at the right point, address what the caller actually said, or does it repeat information the caller already heard? A fast handoff and a clean recovery are graded separately because an agent can do one well and still fail the other.
Yes. Tessary grades production call traces turn by turn and reads the trend in those verdicts over time, so a real shift in how often calls go badly separates from an imperfect grader’s usual noise. Once the trend breaks, Tessary checks what moved upstream of your code: an ASR provider pushing a model update, a telephony carrier changing how it handles pauses, or a voice model getting swapped in behind an API that never changed its name. A prompt edit is one candidate too, but on a voice agent it’s often not the one.
No. Call traces land through Tessary’s ingest path after the agent has already answered the caller, so grading runs alongside the response instead of inside the latency budget the agent is trying to protect. Turn-taking speed is one of the things graders measure, but measuring it doesn’t add to it.
Keep exploring

Related pages
Get started

Find out why callers keep getting cut off.

Connect your voice agent's production call traces at no cost, and find out whether last month's ASR vendor swap is why interruption handling got worse.

Find what broke your agentGrading runs alongside your agent's response, not inside its latency budget.