Why isn't a single bad run enough to call it a regression?

Because agent output is nondeterministic: the same input can produce a fine answer one time and a bad one the next, with nothing about the agent having changed in between. One failure is consistent with that ordinary variance, so it carries almost no information on its own. What makes something a regression is a shift in the distribution, the share of runs that fail or the average of a quality score moving after a change, not before it. That’s why detection compares two populations of runs rather than reading a single verdict: a cohort from before the suspected change against a cohort from after it, on the same kind of input. This holds even when the judgment behind each verdict is itself imperfect, because an imperfect grader’s error rate stays roughly constant run to run, so a rate that doubles or triples is a real shift even when no one verdict can be trusted alone.