# Does a more expensive model make an agent more reliable?

Not in ReliabilityBench, where the model that cost 82 times more was no more reliable. GPT-4o scored 90.42% on the benchmark's pass-both-of-two-tries measure and Gemini 2.0 Flash 91.04%. On clean tasks they scored 95.00% and 96.88%, and on heavily reworded tasks 88.75% and 88.12%.

The bill did differ: $9.77 for GPT-4o's 480 episodes against $0.12 for Gemini's, at the paper's December 2024 list prices of $2.50 and $10.00 per million input and output tokens for GPT-4o, and $0.075 and $0.30 for Gemini.

Price didn't buy reliability for these two models on these tasks, though [routing to a cheaper model can still cause a silent failure](/answers/silent-failures/can-routing-a-turn-to-a-cheaper-model-cause-a-silent-failure) in other setups. [Measuring your own agent's consistency](/answers/reliabilitybench/how-do-you-measure-whether-an-agent-succeeds-consistently) settles it for your tasks. The test itself covers two models and 20 tasks, and the author lists broader model coverage as future work, so it can't say a bigger model never helps.

---

Sources:
- Aayush Gupta, "ReliabilityBench" (arXiv:2601.06112), Table 5 and Section 5.7: https://arxiv.org/abs/2601.06112 (fetched 2026-10-27)

Source: https://tessary.ai/answers/reliabilitybench/does-a-more-expensive-model-make-an-agent-more-reliable
More on Reliabilitybench: https://tessary.ai/answers/reliabilitybench
From Tessary, agent reliability for AI agents in production: https://tessary.ai