Agent Reliability

Detect agent behavior drift

LLM agent behavior drift is when production behavior diverges from design over time, with no single change triggering an alert. Tessary reads the trend across grader verdicts on your live traffic, and when the trend moves, it traces the drop to the change that caused it.

Find what broke your agentSee the SDK and ingest options
The Problem

Drift has no single trigger

Every major framework upgrade can shift behavioral baselines, and most visible errors point at infrastructure rather than the agent. Drift lives in the calls that return successfully while tool selection and output quality shift underneath. Eval suites written before the upgrade validate the agent against its prior behavior, not its current one.

2×

Framework adoption nearly doubled year over year, from over 9% of organizations in early 2025 to almost 18% by early 2026

Datadog State of AI Engineering 2026

5%

of all LLM call spans reported an error in February 2026, and 60% of those errors were exceeded rate limits, not agent logic

Datadog State of AI Engineering 2026

0

Threshold alerts fired as drift accumulates slowly across model updates, framework patches, and context window growth
Research findings

Three dimensions of drift in traces

Rath (2026) studied behavioral degradation in multi-agent LLM systems over extended interactions and identified three dimensions where drift appears: semantic drift (progressive deviation from original intent), behavioral drift (emergence of unintended strategies), and coordination drift (breakdown in multi-agent consensus mechanisms). No individual trace fails; the distribution across traces changes.

Rath (2026), arXiv:2601.04170 ↗

Semantic

progressive deviation from original intent

Behavioral

emergence of unintended strategies

Coordination

breakdown in multi-agent consensus mechanisms
What Tessary Does

Detect the drift, name the cause

Detection is only the first half. When a trend moves on live traffic, the job is naming which change moved it.

Graders drafted from behavior

Tessary drafts graders from your agent's observed production behavior. You correct them rather than write them from scratch, and they sharpen as bugs are fixed and edge cases are shared.

Trend detection over verdicts

Detection reads the trend across grader verdicts over time, not any single score. An imperfect grader misfires at a stable rate, so a failure rate that climbs from 3% to 11% signals a real change even when no threshold trips.

Root-cause attribution

When a trend drops, Tessary traces the failing quality dimension back through the agent's chain to the change that caused it: a prompt edit, a tool update, a dependency bump, an upstream agent's shift, or a model provider update that never touched your repo.

Alerts routed where you work

Alerts route to Slack, Sentry, Linear, PagerDuty, or a webhook, wherever your team already watches production, with the suspected cause attached.

Pre-deploy trouble report

Before a merge, the same attribution engine reads the PR diff and maps it to the failure families it is most likely to trigger: tool-call errors, hallucinated citations, refusal spikes. Changes that ship with no PR at all, a provider update or an upstream shift, still surface through the production trend.

Start from the traces you have

Already on Langfuse or Braintrust? Pull a slice of your production traffic on demand, no SDK changes. When a cause hunt hits a gap in the traces, that gap is the next node to instrument, and the next degradation is easier to trace.
How It Works

From traces to a named cause

Four steps from connecting a trace source to an alert that says what changed.

  1. Connect your trace source

    Connect Langfuse, Braintrust, or the Tessary SDK at evals.tessary.ai. Existing traces work as they are; nothing about the agent changes on day one.

  2. Correct the drafted graders

    Tessary reads your traces and drafts graders for the behaviors that matter. You review and correct them, or add one in plain language: "agent selecting fallback tool on primary task type."

  3. The trend is read over verdicts

    Every new trace is scored as it arrives. Tessary watches how the verdicts move over days and weeks, so drift that accumulates slowly still surfaces.

  4. A drop comes back with its cause

    When the trend moves, the alert that reaches your team names the change behind it, including the causes that never show up in a diff.
Two Approaches

Eval suites vs. production signals

Static eval suites catch regressions against a fixed baseline. Drift happens between eval runs, on live traffic, from causes outside the eval suite's scope.

Static eval suites

Pre-deploy test runs against a fixed baseline

  • Validates against the baseline at the time the suite was written
  • Runs on synthetic or golden datasets, not live traffic
  • Misses drift caused by model provider updates and framework patches
  • No signal on accumulating context window or prompt ambiguity shifts
  • Users notice degradation before the next eval run catches it
Tessary

Tessary production graders

Trend reading and attribution on real traffic

  • Graders drafted from observed behavior run on real production traces
  • Reads the trend across verdicts, so slow drift surfaces without a threshold trip
  • A drop is traced to the prompt edit, framework bump, or provider update behind it
  • The same attribution runs pre-merge on the PR diff
  • Works from the traces your observability stack already holds
FAQ

Answered, plainly.

The main sources are model provider updates, framework version bumps, context window growth, and gradual prompt specification ambiguity. Each one shifts the gap between the agent's intended behavior and what it does in production. The shifts accumulate slowly, so no single event triggers an alert.
A broken agent fails visibly: an error fires, a response is null, a threshold is crossed. Drift is slower. The agent keeps responding, but its tool selection, output coherence, or reasoning path changes over time. Users notice the degradation before standard monitoring does.
Tessary scores every production trace against graders drafted from the agent's observed behavior, then reads the trend across the verdicts over time rather than waiting for a single score to cross a threshold. Drift that accumulates slowly surfaces as a moving trend, even when individual graders are imperfect.
Yes, that is the point of the product. When a grader trend drops, the failing quality dimension is traced back through the agent's chain to the input that fed it, as far back as the traces allow. Commit lineage covers changes that shipped through your repo; trace history covers the ones that did not, like a model provider update or a tool returning different data.
Keep them. Langfuse and Braintrust hold your traces and track scores, and Tessary reads those same traces without re-instrumentation. What they tell you is that a score moved. Tessary tells you which change moved it.
Static suites catch regressions against a fixed baseline, but drift happens between eval runs, on live traffic, often from causes no suite was written for. Graders running on production traces catch the shift, and the trend across their verdicts moves even when every individual run looks acceptable.
Keep exploring

Related pages
Get started

Catch drift before your users do.

Connect the traces you already have and Tessary starts reading the trend. The next time it moves, you get the cause, not just the alert. No credit card required.

Find what broke your agentConnects to Langfuse and Braintrust on day one.