What is fault injection for AI agents?

Fault injection is deliberately breaking a system a piece of software depends on, on purpose, during a test, instead of waiting for it to break in production. Applied to an AI agent, that means running the same task while the tools underneath it time out, rate-limit, or return incomplete data, and measuring how much task success drops under each fault rather than assuming a tool that works today will keep working.

The point is that an agent’s own tools are outside its control the same way a network or a downstream service is outside any distributed system’s control, so testing against a clean environment only measures what the agent can do when nothing goes wrong, which production won’t guarantee. ReliabilityBench builds this into a benchmark score directly, injecting the failure shapes tools actually throw and reporting the pass rate under each one, and found the drop is uneven across fault types, not a flat penalty for any failure. The ways a single tool call can fail is the list a fault-injection suite works from, since you can’t inject a failure shape you haven’t enumerated first.