# How is tau-bench scored?

Tau-bench checks whether the account ended up in the state a correct run would leave it in, the refund issued, the subscription canceled, and nothing else changed alongside it, and ignores how the conversation sounded getting there. A polite, coherent transcript that makes the wrong account change still fails.

The paper's second departure from a normal benchmark is running each task more than once against the same policy and simulated customer, then reporting pass^k: the share of tasks that succeeded on every one of k tries, not pass@1's rate for a single attempt. That matters because a real customer only gets the one attempt the agent actually gives them, not the best of several, so a headline pass@1 number can hide how often the same agent would fail on a retry. [ReliabilityBench applies the identical pass^k metric](/answers/reliabilitybench/how-do-you-measure-whether-an-agent-succeeds-consistently) to tool-using agents generally, not just one customer-service domain.

---

Sources:
- τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (arXiv:2406.12045): https://arxiv.org/abs/2406.12045 (fetched 2026-09-19)

Source: https://tessary.ai/answers/tau-bench/how-is-tau-bench-scored
More on Tau bench: https://tessary.ai/answers/tau-bench
From Tessary, agent reliability for AI agents in production: https://tessary.ai