# Regression detection

A regression is a drop in agent quality caused by a change. Regression detection is the practice of
noticing that drop by comparing the agent's outcome after a change against its outcome before.

Two properties of agents make this harder than in conventional software. First, agent output is non-
deterministic, so a single bad run carries little information. The signal is distributional: a
failure rate or quality score that shifts after a change ships. Second, the change that causes a
regression can come from anywhere in the system around the model. A refactor that changes what
context gets assembled, a dependency bump that alters tool output formatting, or a config edit that
changes retry behavior can each regress the agent's behavior.

There's also a judgment problem the statistics alone can't settle. Teams change their agents on
purpose, and an intended improvement moves the same numbers a defect does. Whether a detected shift
counts as a regression depends on what the agent was meant to do after the change.

## Questions answered under this concept

- [Can a production regression happen without a code change?](https://tessary.ai/answers/regression-detection/can-a-regression-happen-without-a-code-change)
- [Does a passing unit test rule out a regression?](https://tessary.ai/answers/regression-detection/does-a-passing-unit-test-rule-out-a-regression)
- [How do you detect an agent regression after it's already in production?](https://tessary.ai/answers/regression-detection/how-do-you-detect-a-regression-after-deployment)
- [Is every quality shift after a change a regression?](https://tessary.ai/answers/regression-detection/is-every-quality-shift-a-regression)
- [What regressions will a CI gate never catch?](https://tessary.ai/answers/regression-detection/what-a-ci-gate-cant-catch)
- [Why isn't a single bad run enough to call it a regression?](https://tessary.ai/answers/regression-detection/why-isnt-one-bad-run-a-regression)

---

Source: https://tessary.ai/answers/regression-detection
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai