How do I tell which deploy was live when a session failed?

Match the session’s own timestamp against your deploy history: the version live at that moment is whichever one’s deploy window contains the time the session ran, not whichever version is live now. This only works if you can reconstruct that window, so it depends on keeping a deploy log or commit history with real timestamps, not just a changelog of what shipped eventually. Once you have the version, you have the actual code, prompt, and configuration that produced the failing session, which is the input every other step of the investigation needs: you can’t test whether a candidate change explains a failure without knowing what was actually running when it happened. Without this, timing gets fuzzy fast; two changes shipped an hour apart can both look plausible, and only the deploy window tells you which one was actually live when the failure happened.

keep reading

More on this.

How do I find out which change caused my agent's quality drop?Pull the traces where quality dropped into one cohort, line the onset up against your change history, and test each candidate until only one explains what the traces show.answer →What evidence should a regression investigation include?The named cause, the failing traces and findings that support it, and the hypotheses you checked and ruled out, not just a summary of what changed.answer →What comes back when the cause of a failure cannot be established?Sometimes nothing does. An honest attribution says no change explains the failure rather than naming the closest guess, and it keeps what it ruled out.answer →Everything on cause attributionThe definition, and every question answered under cause attribution.concept →

Send us the traces you already emit.

book a demo