Is every quality shift after a change a regression?

No. A regression is a drop the team didn’t want, and the exact same movement in a metric can be what a change was meant to do. Swap in a cheaper model on purpose and pass rate on a hard grader might fall a little; that’s an accepted tradeoff, not a regression, if the team chose it knowing the cost. Tighten a refusal policy and a false-refusal rate climbs; that’s the point of the change, not a side effect of it. Statistics alone can’t settle this, because a shift and a regression look identical in the numbers: both are the distribution moving after something shipped. What decides it is knowing what the agent was meant to do after the change, and checking the shift against that intent rather than against the old baseline by itself. The number alone can’t make that call; someone who knows what the change was supposed to do has to.