# Automationbench

AutomationBench is Zapier's test of whether an AI agent can finish business work that spans
several apps. Zapier released it in April 2026. Each task sets up a small simulated company
across 47 apps, with CRM records, inbox threads, calendars, and spreadsheets, and gives the
agent one request. The agent works alone and finds the right API calls itself.

The score counts only what the agent left behind. A task passes when every fixed check on the
apps' final state holds, including checks that nothing went to the wrong people. The agent's
own reply counts for nothing, and no LLM judge is used. When it launched, every frontier model
scored under 10%. As of October 2026, on version 1.0.6, the top score is about 51%.

Its strength is that it grades outcomes, not claims, and its tasks hide the traps real work
has: near-duplicate records, stale spreadsheet rows, and rules buried in an email. Its
weakness is that the apps are simulations and the tasks are synthetic, so the authors name a
risk of unrealistic or impossible tasks. The 600 public tasks are on GitHub. The leaderboard
uses a harder private set that Zapier sometimes makes harder between versions, so scores
don't compare across versions.

## Questions answered under this concept

- [Can I run AutomationBench on my own agent?](https://tessary.ai/answers/automationbench/can-i-run-automationbench-on-my-own-agent)
- [Does a high AutomationBench score mean an agent will work in production?](https://tessary.ai/answers/automationbench/does-a-high-automationbench-score-mean-an-agent-will-work-in-production)
- [How is AutomationBench different from tau-bench?](https://tessary.ai/answers/automationbench/how-is-automationbench-different-from-tau-bench)
- [Is there an independent leaderboard for AutomationBench?](https://tessary.ai/answers/automationbench/is-there-an-independent-leaderboard-for-automationbench)
- [What changed in the latest version of AutomationBench?](https://tessary.ai/answers/automationbench/what-changed-in-the-latest-version-of-automationbench)
- [What does an AutomationBench score mean?](https://tessary.ai/answers/automationbench/what-does-an-automationbench-score-mean)
- [What is AutomationBench?](https://tessary.ai/answers/automationbench/what-is-automationbench)
- [Which kinds of business work do AI agents do worst at on AutomationBench?](https://tessary.ai/answers/automationbench/which-business-functions-do-ai-agents-do-worst-at-on-automationbench)
- [Why do AI agents say a task is done when it failed?](https://tessary.ai/answers/automationbench/why-do-ai-agents-say-a-task-is-done-when-it-failed)

---

Source: https://tessary.ai/answers/automationbench
All concepts: https://tessary.ai/answers
From Tessary, agent reliability for AI agents in production: https://tessary.ai