# Silent failures

A silent failure is an agent run that completes normally and produces a wrong result. The run
returns a success status, throws no exception, and the response reads fluently, but the user didn't
get what they asked for. The agent misread a tool result, claimed a task was done when it wasn't,
answered from stale data, or drifted from its instructions partway through.

The failure is silent because standard monitoring measures properties of the run, and the problem is
in the content of the response. Error rates, latency, and uptime all stay healthy while it happens.
The first person to notice is usually a user, and a user report arrives some time after the failure
started, so by then it has usually happened many times.

Silent failures are typically low-probability, and that determines how long they survive. A failure
that hits 2% of sessions, observed through a 5% review sample, surfaces in about one session in a
thousand, so evidence accumulates slowly. A single occurrence is also hard to distinguish from
noise: one odd trace could be anything, and the pattern only becomes visible once many traces have
failed the same way.

## Questions answered under this concept

- [How do you detect silent failures in production?](https://tessary.ai/answers/silent-failures/how-do-you-detect-silent-failures-in-production)
- [How many traces do you need before you're likely to catch a rare failure once?](https://tessary.ai/answers/silent-failures/how-many-traces-before-you-catch-a-rare-failure)
- [What is a silent failure in an AI agent?](https://tessary.ai/answers/silent-failures/what-is-a-silent-failure-in-an-ai-agent)
- [Why does a trace look normal when a tool returns stale data?](https://tessary.ai/answers/silent-failures/why-does-a-trace-look-normal-when-a-tool-returns-stale-data)
- [Why doesn't monitoring catch a silent agent failure?](https://tessary.ai/answers/silent-failures/why-doesnt-monitoring-catch-a-silent-agent-failure)
- [Would stratified sampling fix a silent failure?](https://tessary.ai/answers/silent-failures/would-stratified-sampling-fix-a-silent-failure)

---

Source: https://tessary.ai/answers/silent-failures
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai