Why do AI agents hallucinate?

A language model generates the most probable next word, not a verified one. Nothing in that process distinguishes a sentence built from retrieved facts from a sentence built from the model’s own prior, so both come out equally fluent. Left alone, the model fills a gap in its sources the same way it fills a gap in a sentence: with whatever completes the pattern.

Retrieval is meant to close that gap by handing the model the facts before it answers, but retrieval can fail three separate ways. The right document never gets fetched, so there’s nothing to ground the answer. It gets fetched but the model answers from its prior anyway. Or it gets fetched and used, but misquoted along the way. None of those breaks the model’s fluency, which is exactly why a hallucination reads the same as a correct answer until someone checks it against the source.