How do I tell whether an eval case has gone stale?

Check the date the case was captured against what’s changed since. A case is stale once the prompt, tool, or model it was written against has moved and nobody has re-verified that the expected behavior still holds. The case still runs and still produces a score, which is exactly what makes staleness dangerous: nothing errors, the number just quietly stops meaning what it used to.

You can’t catch this without a capture date on the case in the first place, which is why recording where a case came from matters as much as writing the case. In practice, that means periodically re-pulling cases against current traffic and current agent behavior rather than trusting a set once and leaving it alone. A case that no longer matches how the agent behaves isn’t testing the agent anymore, it’s testing agreement with its own past.