Does a high BFCL score mean an agent will work in production?

Not fully. BFCL scores whether a model picks the right function and fills the right arguments against a fixed set of curated requests, a controlled setting where the correct call is known in advance. Your production traffic won’t phrase requests the way a benchmark does, and your own tool definitions carry quirks BFCL never tested against.

The 70% of the V4 score that’s Multi-Turn and Agentic doesn’t close that gap either. Both categories run against a handful of mock backends Berkeley built for the benchmark, a file system, a vehicle-control system, a trading bot, a travel-booking system, each with a known starting state and a known correct end state. None of them ever time out, rate-limit, or hand back a malformed result, so a model can top those categories without once having handled a tool call that actually failed.

sources

keep reading

More on this.

What changed in BFCL's V4?V4 cut single-turn testing from two-thirds of the score to a fifth and gave the freed weight to a new Agentic category: web search, memory, and format sensitivity.answer →What is BFCL?BFCL, the Berkeley Function Calling Leaderboard, tests whether a model picks the right tool, fills in the right arguments, and calls nothing when nothing fits.answer →Everything on bfclThe definition, and every question answered under bfcl.concept →

Self-host Tessary.

Free and open source. Point it at the traces your agent already emits.

Self-host Tessary for me by following https://github.com/tessaryai/tessary/blob/main/setup.md

docker compose -f oci://docker.io/tessaryai/tessary:compose up -d -y