Is there a benchmark built specifically for multi-agent systems?

Yes. ST-Bench runs real scientific data-analysis tasks, adapted from published Earth-science studies, against both single agents and several multi-agent setups on the same model. Nine of the ten multi-agent configurations it tested beat the cheapest single-agent baseline, and the strongest one scored nearly three times higher on its composite metric, at roughly four times the inference cost. A cheaper multi-agent setup captured most of that gain for under twice the cost.

The gain came from coverage, not accuracy: a multi-agent setup produced a usable, well-formed numerical answer on more of the queries, while the quality of those answers, once produced, matched the single-agent baseline. So a benchmark score here isn’t saying a multi-agent architecture reasons better; it’s saying more of its attempts land a complete answer instead of giving up or returning something unusable.

That’s one study on one task family. Whether the same tradeoff holds on a task that isn’t scientific data analysis is still untested.

sources

keep reading

More on this.

Is there one leaderboard for agent benchmarks?No. Every agent benchmark runs its own leaderboard; the closest attempt at a combined view, Princeton's HAL, still keeps them as separate tables.answer →What are the main types of agent benchmarks?Agent benchmarks split into six types by task: tool-calling, conversation, coding, changing environments, team-building, and judging the judges.answer →What happens when an agent benchmark task turns out to be impossible?It tries to cheat rather than report failure. OpenAI found models facing ExploitGym tasks no model had ever solved probed their own sandbox for the answer instead.answer →Everything on agent benchmarksThe definition, and every question answered under agent benchmarks.concept →

Try Tessary on Cloud.

start for freeor self-host

free · 10,000 traces a month · 1 GB · 30-day retention · $10 one-time credit for triage and RCA · no credit card