# Is there an independent leaderboard for AutomationBench?

Yes: Artificial Analysis runs an independent leaderboard, AutomationBench-AA, launched July 6, 2026 in partnership with Zapier, against a private 657-task subset the company doesn't publish anywhere else. The tasks span six categories, finance, HR, marketing, operations, sales, and support, across 40 simulated app environments including Gmail, Salesforce, and Jira.

The headline score is the share of objectives an agent completes without breaking a guardrail, out of nearly 12,000 assertions across the suite: each one is either an objective the agent must achieve or a guardrail that already passes at the start and must stay that way. Each task runs once, capped at 50 turns, and grading checks whether the right data landed in the right system rather than whether the agent's reasoning looked sound.

At launch, Claude Fable 5 led at 48.6%, just ahead of Claude Opus 4.8 at 48.5%, with Gemini 3.5 Flash and GPT-5.5 both in the low 40s and GLM-5.2, the top open-weights model, at 27.8%. [A single run says nothing about how often that same agent would pass the same task again](/answers/reliabilitybench/how-do-you-measure-whether-an-agent-succeeds-consistently), which AutomationBench-AA doesn't test either.

---

Sources:
- Artificial Analysis, "Announcing Zapier AutomationBench on Artificial Analysis": https://artificialanalysis.ai/articles/announcing-zapier-automationbench-aa (fetched 2026-10-21)

Source: https://tessary.ai/answers/automationbench/is-there-an-independent-leaderboard-for-automationbench
More on Automationbench: https://tessary.ai/answers/automationbench
From Tessary, agent reliability for AI agents in production: https://tessary.ai