Why do AI agents hallucinate?

A language model generates the most probable next word, not a verified one. Nothing in that process distinguishes a sentence built from retrieved facts from a sentence built from the model’s own prior, so both come out equally fluent. Left alone, the model fills a gap in its sources the same way it fills a gap in a sentence: with whatever completes the pattern.

Retrieval is meant to close that gap by handing the model the facts before it answers, but retrieval can fail three separate ways. The right document never gets fetched, so there’s nothing to ground the answer. It gets fetched but the model answers from its prior anyway. Or it gets fetched and used, but misquoted along the way. None of those breaks the model’s fluency, which is exactly why a hallucination reads the same as a correct answer until someone checks it against the source.

keep reading

More on this.

How can an answer be faithful to its context and still be wrong?Faithfulness only checks the answer against what was retrieved, so an answer built entirely from a partial or unrepresentative slice of the source passes even though it's wrong.answer →What's the difference between faithfulness, groundedness, and context presence?They check three different points in the same pipeline: was the needed source retrieved, did the model actually use it, and did it represent that source accurately once it did.answer →How do I catch agent hallucinations when my error rate stays flat?Error rate tracks whether a run crashed, not whether its answer was true, so you need a check that compares the answer's content against its actual sources instead.answer →Everything on hallucinationsThe definition, and every question answered under hallucinations.concept →

Send us the traces you already emit.

book a demo