What is a silent failure in an AI agent?

A silent failure is an agent run that completes normally and gets the answer wrong.

The term predates agents. In any system, a silent failure is one with no error signal attached: the process exits 0, the request returns 200, no exception is thrown, and the wrong thing happened anyway. What makes agents produce them constantly is that the output is prose. Fluent text passes every check a status code can perform, and nothing about the way an answer reads tells you whether it is true.

The trace shape is ordinary. A tool returns a record that was correct last week, the agent reads it, summarizes it in a paragraph the user has no reason to doubt, and the span closes with status OK. No layer reported a problem, because none of them was checking the record against anything.

That is why standard monitoring never sees one, and why detection means reading what the agent said.

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y