How do I catch a regression in a Claude Agent SDK agent?

There’s no built-in regression feature; you assemble one from two things the SDK already gives you. A PostToolUse or Stop hook runs your own check at the end of a turn and can record a pass or fail without touching the agent’s prompt, and a session can be resumed or forked, which means a fixed case can be replayed from the exact state it started in against a changed prompt, tool set, or model, rather than a fresh run that might diverge for unrelated reasons.

Keep a small set of cases as sessions worth re-forking, run each one through the hook before and after a change, and compare the pass rate rather than eyeballing individual transcripts. A drop names which case regressed, the same discipline any agent needs to catch a regression, just built from the SDK’s own hooks and forkable sessions instead of a separate harness.

sources

keep reading

More on this.

Two ways to run Tessary.

Tessary is an open-source agent reliability platform. Cloud and self-hosted run the same workflow on the OpenTelemetry traces your agent already emits.

Tessary Cloud

We host it for you. Send your first trace with nothing to deploy and no model key.

what's includedper organization
traces
10,000 per calendar month
stored trace data
1 GB
retention
30 days
model credit
$10, one-time, for triage and root-cause analysis
credit card
not required

Self-hosted Tessary

Run the open-source code on your own infrastructure with one command. Add your own model key for triage and root-cause analysis.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y