What is Vending-Bench?

Vending-Bench is Andon Labs’ benchmark for whether an AI agent stays coherent over a very long task: the agent runs a simulated vending-machine business for one simulated year, starting with $500, and is scored on its bank balance at the end. It finds suppliers, orders stock by email, restocks the machine, sets prices, and pays a $2 daily location fee.

As of October 2026 the current version is Vending-Bench 2, released November 18, 2025. It replaced the original from Axel Backlund and Lukas Petersson’s February 2025 paper, “Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents,” which Andon Labs now lists as deprecated. Version 2 added problems Andon Labs says it learned from its real vending-machine deployments: suppliers that try to exploit the agent, late deliveries, suppliers going out of business, and customers demanding refunds. Vending-Bench Arena, Andon Labs’ first multi-agent version, puts several agents’ machines at one location, where they can email, trade with, and undercut each other, each scored on its own balance.

Unlike a typical agent benchmark, it has no list of tasks to pass or fail. A run is one year-long task of 3,000 to 6,000 messages, per Andon Labs’ Vending-Bench 2 page, the suppliers are other language models, and customer sales are simulated from the day of the week, season, weather, and price.