# What changed in the latest version of Vending-Bench?

Vending-Bench 2 clarified the scoring criteria so agents know exactly what they are optimized for, and added planning tools, specifically note-taking and reminder systems, neither of which the leaderboard's dollar figure shows, per Andon Labs' Vending-Bench 2 page. The original left the goal to the agent's own reading of how the simulation reacted; Andon Labs rewrote it so that's no longer a judgment call.

The planning tools matter because the agent's context window is capped at roughly 69,000 tokens and trims to about 61% of messages once that fills, per the benchmark's own system prompt, the same [cutoff that can erase an earlier instruction](/answers/failure-modes/what-happens-past-the-context-window) in any long-running agent. A supplier promise made in month two has to survive that trim to matter in month six, and before this version the agent had nowhere dedicated to keep it.

Neither change moves the number the leaderboard prints; both change whether that number reflects the same test for every model it runs.

---

Sources:
- Vending-Bench 2 (Andon Labs): https://andonlabs.com/evals/vending-bench-2 (fetched 2026-10-26)

Source: https://tessary.ai/answers/vending-bench/what-changed-in-the-latest-version-of-vending-bench
More on Vending bench: https://tessary.ai/answers/vending-bench
From Tessary, agent reliability for AI agents in production: https://tessary.ai