Which kinds of business work do AI agents do worst at on AutomationBench?

HR and Support. On Zapier’s AutomationBench leaderboard, version 1.0.6, the best any model scores in HR is 39.17%, from Google’s Gemini 4 Argon (High), and in Support 45.00%. Operations is the easiest function at 68.00%, then Sales at 58.12%, Marketing at 58.00%, and Finance at 50.00%. Finance is the one function where Gemini doesn’t lead: Anthropic’s Claude Sonnet 5.5 (Max) does.

Each figure is the top model’s score in that function at its best effort level, so these are ceilings, not averages. The runner-up in HR scores 31.67%, so the gap is not one model’s weakness. Zapier’s own failure analysis points at how agents fail, not at which function they fail in: in 72% to 91% of failures, depending on the model, the agent reported success while the final state was wrong.

The tasks are synthetic, built from Zapier workflow patterns, so a function’s score describes agents on those simulated companies, not on your own HR system.

sources

keep reading

More on this.

Why do AI agents say a task is done when it failed?An agent's "done" is generated text that nothing checks against the systems it changed unless someone builds that check, so failed runs still report success.answer →Can I run AutomationBench on my own agent?Yes, through Zapier's own CLI on the 600 public tasks, but the score won't match the leaderboard, which grades a separate, harder, private task set.answer →Does a high AutomationBench score mean an agent will work in production?Not on its own: Zapier's AutomationBench scores one run of one instruction against simulated apps that, by Zapier's own account, don't match production behavior.answer →Everything on automationbenchThe definition, and every question answered under automationbench.concept →

Try Tessary on Cloud.

start for freeor self-host

free · 10,000 traces a month · 1 GB · 30-day retention · $10 one-time credit for triage and RCA · no credit card