# Why can't I just increase the context window instead of using RAG?

A longer context window helps, but it doesn't solve the problem RAG solves. Even a million-token window doesn't hold a company's full documentation, ticket history, and product catalog at once, and every token you paste in gets billed and processed on every call, whether that document was needed or not.

Stuffing a window full of documents also degrades retrieval within the prompt itself. Research on long-context models found they're measurably worse at using information buried in the middle of a long context than information near the start or end (Liu et al., "Lost in the Middle"). A bigger window filled with mostly irrelevant text can make a model perform worse than a smaller window filled with exactly the chunks that matter.

The two aren't opposites. A bigger window makes each retrieved chunk cheaper to include, but it doesn't remove the need to find the right ones first.

---

Sources:
- Lost in the Middle: How Language Models Use Long Contexts (Liu et al.): https://arxiv.org/abs/2307.03172 (fetched 2026-08-25)

Source: https://tessary.ai/answers/retrieval-augmented-generation/why-cant-i-just-increase-the-context-window-instead-of-using-rag
More on Retrieval augmented generation: https://tessary.ai/answers/retrieval-augmented-generation
From Tessary, agent reliability for AI agents in production: https://tessary.ai