How can an answer be faithful to its context and still be wrong?

Faithfulness only checks the answer against what was retrieved, not against everything the source actually says. An answer built entirely from a partial slice of a document, half a policy, one clause of a contract, one turn of a longer conversation, can accurately represent that slice and still land on the wrong conclusion, because the part that would have changed the answer never made it into context.

That’s a context-presence failure wearing a faithfulness pass. The model didn’t misquote anything; it correctly summarized what it was given, and what it was given wasn’t enough. This is why the three checks have to run separately rather than one score standing in for all of them: a grader that only asks “does the answer match the context” will wave this case through, because by that narrow question, it did.

keep reading

More on this.

What's the difference between faithfulness, groundedness, and context presence?They check three different points in the same pipeline: was the needed source retrieved, did the model actually use it, and did it represent that source accurately once it did.answer →How do I catch agent hallucinations when my error rate stays flat?Error rate tracks whether a run crashed, not whether its answer was true, so you need a check that compares the answer's content against its actual sources instead.answer →What is a hallucination in an AI agent?An agent stating something its sources don't support, whether it misquotes a document it retrieved, ignores that document entirely, or never retrieved anything to begin with.answer →Everything on hallucinationsThe definition, and every question answered under hallucinations.concept →

Send us the traces you already emit.

book a demo