How do I tell which deploy was live when a session failed?

Match the session’s own timestamp against your deploy history: the version live at that moment is whichever one’s deploy window contains the time the session ran, not whichever version is live now. This only works if you can reconstruct that window, so it depends on keeping a deploy log or commit history with real timestamps, not just a changelog of what shipped eventually. Once you have the version, you have the actual code, prompt, and configuration that produced the failing session, which is the input every other step of the investigation needs: you can’t test whether a candidate change explains a failure without knowing what was actually running when it happened. Without this, timing gets fuzzy fast; two changes shipped an hour apart can both look plausible, and only the deploy window tells you which one was actually live when the failure happened.

keep reading

More on this.

What evidence should a regression investigation include?The named cause, the failing traces and findings that support it, and the hypotheses you checked and ruled out, not just a summary of what changed.answer →What comes back when the cause of a failure cannot be established?Sometimes nothing does. An honest attribution says no change explains the failure rather than naming the closest guess, and it keeps what it ruled out.answer →What is cause attribution?Cause attribution is the investigation that names the specific change responsible for a regression, once detection has established that one happened.answer →Everything on cause attributionThe definition, and every question answered under cause attribution.concept →

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required
start for free

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y