What is agent reliability?

Agent reliability is how consistently an agent’s production behavior matches what it was built to do, measured across the inputs it actually receives and over time. It’s a property of the whole system, not just the model: behavior depends on the model, the prompts, the tools it can call, and the data it reads, and a change to any one of those shifts it. Agents are also nondeterministic, so the same input can produce different outputs on different runs. That means reliability isn’t a pass or fail on one run; it’s a statement about the distribution of behavior across many runs. A reliable agent keeps that distribution stable as everything around it changes. An unreliable one drifts, usually quietly: it keeps producing fluent text and completed tool calls, so a wrong decision looks the same on the surface as a right one.