Why does an AI model make up an answer instead of saying it doesn't know?

Because the benchmarks and tests it’s trained and graded against score a confident guess the same as a right answer, and score “I don’t know” as a guaranteed loss. A 2025 paper on why language models hallucinate frames this as a scoring problem: most evaluations grade output as strictly right or wrong, so a model that always guesses when unsure scores higher on average than one that abstains, even though the guesser is wrong more often. Training and leaderboards both optimize for exactly that kind of test-taking.

This is a different cause from a bad retrieval step or a stale document. Even a model with perfect access to its sources will pad a gap with an invented fact if nothing in how it’s measured rewards saying it doesn’t know. The paper’s proposed fix isn’t a better hallucination detector, it’s changing how the benchmarks that dominate leaderboards are scored, so admitting uncertainty stops being the losing move.