What is Tessary's tool_error classifier?

tool_error watches the rate at which a tool’s calls fail, not any single call. A call counts as failed only on structural evidence: an error status on the span, a recorded exception, or a result that declares itself an error. Output text that merely mentions the word error never counts.

A detection is one finding per tool, and it carries the tool’s name, when the rate moved, which failure patterns drove it, and which call sites the traffic came through. From there it goes the way any classifier’s finding does.

Two limits. Because the evidence is structural, a tool that returns success with a wrong result is invisible to this check, which is a different failure needing a different one. And the per-tool alert threshold comes from a false-alarm budget that has never been measured against real traffic.

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y