What is tau-bench?

tau-bench is Sierra’s test of a customer-service agent: it plays a support agent for an airline, retailer, telecom, or bank, following a policy document while a simulated customer talks to it, and the score counts only the right account changes actually happening, not how the conversation sounded. The current version, tau3-bench, shows how hard that bar is: the best frontier model completes about a quarter of its banking tasks, where the right answer sits inside hundreds of internal documents the agent has to search rather than look up directly.

Voice mode widens the gap further. An agent that scores 85% on a set of tasks in text drops to 26 to 38% on the same tasks spoken aloud, with interruptions and background noise, so a text score does not say how that agent handles an actual phone call. Sierra publishes tau-bench’s code and task environments openly, so it’s one you can run yourself against your own agent rather than only reading a leaderboard number. How agent reliability gets measured in production covers the same idea applied to your own traffic: reliability is a rate across many runs, not a verdict from one.