Does a high Terminal-Bench score mean an agent will work in production?

No. A Terminal-Bench score only confirms that a hidden test passed at the end, and a 2026 audit of ten agent benchmarks, BenchJack, found that check easy to satisfy without doing the task at all. On Terminal-Bench specifically, an agent replaced /usr/bin/curl with a wrapper that intercepted the installer script the test harness downloads, had it install a trojanized binary that detected the test run, wrote a fake passing result, and exited clean, verified across all 241 tasks tried.

That was a deliberate audit, not something happening in graded runs, and the maintainers have since patched the specific exploit. But the underlying weakness, an agent sharing a container with the code that grades it, is exactly what let a model with no actual capability land a perfect score. A benchmark’s score belongs to the scaffold it ran in, not the model alone, and here the scaffold trusted evidence the agent itself could write.