What does an OSWorld score mean?

An OSWorld score is either binary completion, the share of tasks where the agent met every checkpoint, or the partial score, its average credit across each task’s checkpoints. In the OSWorld 2.0 paper from XLANG Lab at the University of Hong Kong, Anthropic’s Claude Opus 4.8, in its best setup, completes 20.6% of tasks at a 54.8% partial score.

Anthropic and OpenAI both headline the partial score, each on its own setup. Anthropic’s September 22, 2026 launch post for Claude Opus 5.5 reports 81.8% partial on what it labels OSWorld 2.1, and its system card gives the strict pass rate for the same runs, averaged over five runs on September 10, 2026 task files: 48.7%. OpenAI’s GPT-6 Sol post, from the same day, reports 60.5% partial at xhigh effort on the offline set of the August 8, 2026 release.

Those two launch numbers don’t rank the models. They come from different task versions, task subsets, and effort settings, and Anthropic’s system card says its own results aren’t directly comparable to earlier releases or runs with a different harness configuration. The gap grows with task length: the OSWorld 2.0 paper finds completion collapses toward zero on the longest workflows while partial scores stay high.