What changed in the latest version of OSWorld?

The latest version of OSWorld, as of October 2026, is OSWorld 2.1 from XLANG Lab at the University of Hong Kong, a bug-fix release published September 16, 2026 that keeps OSWorld 2.0’s 108 long-horizon tasks, updates its mock websites, adds new agents, fixes Task015’s grading among other evaluation fixes, and pins every component to one release for reproducible runs.

Most of the change came one step earlier. OSWorld 2.0, released June 26, 2026 in its own repository, replaced the original OSWorld’s 369 tasks from 2024, which took a person a median of about two minutes, with 108 workflows that take a median of about 1.6 hours, per the OSWorld 2.0 paper (arXiv:2606.29537). It also moved the task code out of the public repository into a gated Hugging Face dataset.

Between 2.0 and 2.1, the August 8, 2026 release changed 14 tasks to stop agents from reward hacking them, a problem an audit also found in SWE-bench Pro, and modified 31 tasks for grading robustness and fairness. XLANG treats each release as its own comparable set and says not to mix code, tasks, assets, or websites across them. As of October 2026 XLANG hosts mock websites only for the August release, so running the June release (unhosted since September 8, 2026) or 2.1 means self-hosting them. One known issue remains in 2.1: Task029’s setup can hit a 120-second startup timeout, and the tested fix isn’t in this release.