How is CursorBench scored?

Cursor hasn’t published the correctness rubric; it reports accuracy averaged over repeated attempts at each task, alongside efficiency numbers most benchmarks leave out: completion tokens, end-to-end latency, and inference cost per task, tracked separately because a model that’s barely more accurate but far slower or more expensive isn’t more useful in an editor where a developer is waiting on the response. Models are also compared across several reasoning-effort settings, high, medium, and low, rather than at one fixed setting, since the same model can land at different points on Cursor’s accuracy-versus-cost curve depending on how hard it’s told to think.

That’s a different scoring philosophy from scoring an agent on whether every repeated attempt succeeds, which counts a task only if the model never fails it. Averaging rewards a model that’s usually right over one that’s inconsistent but occasionally perfect, which fits Cursor’s own framing of the benchmark as a signal for interactive developer workflows rather than a strict reliability test.