# Can I run AutomationBench on my own agent?

Yes: AutomationBench's 600 public tasks are free to run yourself through Zapier's own CLI, installed with `uv sync` and run as `uv run auto-bench --model <model>`, but the number you get won't match the official leaderboard.

The public set splits evenly across six domains, sales, marketing, operations, support, finance, and HR, 100 scored tasks each, plus a separate 200-task "simple" domain that doesn't count toward the score. The CLI points at a model through an API key or a custom base URL; its README doesn't describe plugging in your own agent harness, so you're testing a model against Zapier's own scaffolding, not necessarily [the harness you'd actually ship](/answers/agent-harness/why-does-the-same-model-score-differently-in-different-harnesses).

The leaderboard itself grades a separate, held-out private set per domain that Zapier says follows a similar distribution but runs harder, so a local run gives you directional agreement with the leaderboard, not its own number, and there's no documented way to submit a local result for an official score.

---

Sources:
- zapier/AutomationBench README (GitHub): https://github.com/zapier/AutomationBench (fetched 2026-10-19)

Source: https://tessary.ai/answers/automationbench/can-i-run-automationbench-on-my-own-agent
More on Automationbench: https://tessary.ai/answers/automationbench
From Tessary, agent reliability for AI agents in production: https://tessary.ai