What is a deploy gate for an AI agent?

A deploy gate is a check that runs a set of evaluations against a change before it merges, and blocks the merge if the change fails them. For an agent, the change is usually a diff to a prompt, the context it assembles, a tool definition, or a model config, and the gate runs the modified agent against a set of scenarios before anything ships. Each evaluation pairs a scenario, a user request or a situation that calls for a specific tool, with a judgment: comparing the output to a known-good answer, asserting a property like the right tool being called, or having a model grade it against a rubric.

The set grows the way regression tests do. Every evaluation encodes a failure the team decided must stay fixed, so it keeps checking for that failure on every future change, and a red result points at a specific mode that’s happened before. Because a gate reads diffs, it only catches regressions that arrive through a merge, not ones that show up on their own once something ships.