# Osworld

OSWorld is XLANG Lab's benchmark for AI agents that use a computer the way a person does:
reading the screen and working the mouse, keyboard, and apps on a real virtual machine. The
original 2024 version had short tasks that frontier agents now mostly pass. OSWorld 2.0,
released in June 2026, replaced them with 108 long workflows that take a person a median of
about 1.6 hours, and OSWorld 2.1, a September 2026 bug-fix release, is the current version.

Each task is graded against a list of checkpoints, so a result is reported either as the
share of tasks fully completed or as partial credit, and the two can sit far apart. As of
October 2026 no agent fully completes half of the tasks.

Its strength is realism: tasks span several apps and websites, some change while the agent
works, and the task code is kept out of the public repository so agents can't look up the
answers. Its weakness is cost and setup: runs take hundreds of steps, teams self-host the
task websites, and scores from different releases, task subsets, or agent harnesses don't
compare.

## Questions answered under this concept

- [Can an OSWorld task fail because of a bug in the benchmark, not the agent?](https://tessary.ai/answers/osworld/does-an-osworld-task-fail-because-of-a-bug-in-the-benchmark-itself)
- [Is computer use reliable yet?](https://tessary.ai/answers/osworld/is-computer-use-reliable-yet)
- [What's the difference between OSWorld and OSWorld-Verified?](https://tessary.ai/answers/osworld/osworld-vs-osworld-verified)
- [What changed in the latest version of OSWorld?](https://tessary.ai/answers/osworld/what-changed-in-the-latest-version-of-osworld)
- [What does an OSWorld score mean?](https://tessary.ai/answers/osworld/what-does-an-osworld-score-mean)
- [What is a computer-use agent?](https://tessary.ai/answers/osworld/what-is-a-computer-use-agent)
- [What is OSWorld?](https://tessary.ai/answers/osworld/what-is-osworld)

---

Source: https://tessary.ai/answers/osworld
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai