Is agent reliability the same as uptime?

No. Uptime tracks whether the service responded: the process didn’t crash, the request didn’t time out, the span closed without an error status. None of that says whether the agent did the right thing. A tool call that returns success and writes nothing looks identical, on an uptime dashboard, to one that worked. So does an answer that got shorter or wronger after a fallback kicked in. The span still closes clean, because nothing in the transport layer knows the content was bad. Reliability has to look inside the response: whether the tool call actually had the claimed effect, whether the answer used the right facts, whether the agent did what the user asked. A system can run with high uptime and be unreliable in this sense the whole time, because the two numbers measure different layers of the same request.

keep reading

More on this.

Does passing your eval suite before launch mean an agent will stay reliable?No. An eval suite checks the agent against the model and tools as they existed when you ran it, both of which keep changing after launch, so passing once doesn't mean it stays reliable.answer →How do you measure agent reliability in production?As a rate across many runs, such as the share of production sessions that behave as intended, tracked over a rolling window, rather than a pass or fail judged on a single run.answer →What causes an AI agent to become less reliable?Any change to the model, prompts, tools, or data an agent depends on can degrade it, and the model itself can change on the provider's schedule with no deploy on your side.answer →Everything on agent reliabilityThe definition, and every question answered under agent reliability.concept →

Send us the traces you already emit.

book a demo