What changed in the latest version of tau-bench?

The latest version, tau3-bench, released March 18, 2026, added two domains: banking, where the answer sits inside 698 documents across 21 product categories rather than one lookup, and voice, where the same tasks get spoken instead of typed. The best model tested, GPT-5.2 at high reasoning, completes about a quarter of the banking tasks, and a voice agent that hits 85% on the text version of a task drops to 26 to 38% saying it out loud.

The version before it, tau2-bench, changed something more basic: the simulated customer could act on the environment, restarting their phone or turning off airplane mode, not just talk, and it added a telecom domain built around that. The paper reports a significant drop in success once agents had to guide a person instead of just answering one, the same kind of consistency gap ReliabilityBench measures by running a task more than once instead of trusting a single pass.

sources

keep reading

More on this.

What does a tau-bench score mean?A tau-bench score averages domains with very different difficulty, and masks how little a single clean run says about a typical one.answer →What is tau-bench?tau-bench is Sierra's test of a customer-service agent against a simulated customer, scored only on whether the right account changes actually happened.answer →Can tau-bench grade a failing run as a pass?An October 2026 audit found a bug in tau-bench's own reward function that can turn a failed run into a recorded pass, though tau-bench's own runner is unaffected.answer →Everything on tau benchThe definition, and every question answered under tau bench.concept →

Try Tessary on Cloud.

start for freeor self-host

free · 10,000 traces a month · 1 GB · 30-day retention · $10 one-time credit for triage and RCA · no credit card