Does a calm bug report count as frustration?

Usually not: a matter-of-fact bug report reads as neutral, because the classifier flags a user who is unhappy with the assistant, and a calm description of a problem doesn’t show that.

The exception is repetition. A user re-asking after the assistant clearly failed counts, even in a flat tone. “My export button returns a 500” as the opening message isn’t even scored, since a conversation’s first two user messages never are. Typed again after two answers that didn’t fix it, the same sentence can flag.

Frustration and severity are still different signals. A calm report can point at a critical failure, and a furious message can be about something trivial, or about something outside the chat entirely, which never flags. Error-rate monitoring already catches thrown exceptions and failed tool calls. What this adds is the agent that’s technically working and still losing the user.

The limit: when a message is ambiguous, the classifier is told to call it neutral, so it misses some real frustration rather than flag calm users.

keep reading

More on this.

Does Tessary's frustration classifier see the whole conversation, or just the last few turns?It reads the two turns right before the one it's scoring, not the whole conversation, so frustration that hasn't recurred in a few turns won't factor in.answer →Does one frustrated message trigger an alert?No. One flagged message writes a detection, not an alert. A case opens only when a call site's rate of frustrated conversations rises above the rate it learned as normal.answer →How long before Tessary's frustration classifier starts judging a call site?It starts once its reference holds 100 scored conversations, down from 200, and keeps learning until 1,000 instead of freezing right away.answer →Everything on tessary frustrationThe definition, and every question answered under tessary frustration.concept →

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required
start for free

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y