What Is Retrieval-Augmented Generation?
Retrieval-Augmented Generation (RAG) is an architecture that fetches relevant passages from an external source at question time and hands them to a language model before it writes an answer.
Retrieval-Augmented Generation (RAG) is an architecture that fetches relevant passages from an external source at question time and hands them to a language model before it writes an answer. It is not training. Training fixes patterns into model weights months in advance, while retrieval happens live, which is how a model can quote a page published last week.
Why retrieval changes what you optimise
Classic SEO treats the page as the unit of competition. You build one URL, point links at it, and expect the whole thing to rank as a block. Retrieval does not work that way. It scores fragments, so the unit that wins is a passage, and the rest of your page is simply not in the running.
The consequence is uncomfortable if you have spent years building long-form assets. A 4,000-word guide that answers the actual question in paragraph 38, wrapped in throat-clearing, is hard to retrieve cleanly. A 200-word section that states the answer, names the conditions, and stops is easy. Same domain, same authority, different outcome.
Before you rewrite anything, find out which questions already surface you and which return a competitor. A read on where AI answers currently place you tells you which pages are worth restructuring and which are fine as they are.
How retrieval-augmented generation works step by step
The pipeline runs in two halves. One half happens in advance, when documents are prepared and indexed. The other happens in the second or two after someone asks a question. Understanding where the split falls tells you which parts you can influence as a publisher, and which parts are decided inside a system you will never see.
- Content gets chunked. Source documents are split into passages, often a few hundred words each, sometimes on heading boundaries and sometimes on a fixed window that ignores your structure entirely.
- Each chunk becomes a vector. An embedding model converts the passage into a list of numbers representing its meaning, so that similar ideas sit near each other in the same mathematical space.
- The chunks are indexed. Those vectors go into a store built for similarity search, alongside metadata such as the source URL and title.
- The question is embedded too. The user’s query is converted the same way, sometimes after being rewritten into several sub-questions to widen what gets found.
- Candidates are retrieved and reranked. The system pulls the nearest chunks, then a second model reorders them by how directly each one answers the question rather than how topically close it is.
- The model writes from what it was handed. The top passages are placed in the context window with an instruction to answer from them, and the sources behind those passages become the citations.
Two things follow from that sequence. First, a passage the retriever never selects cannot be cited, no matter how strong the domain publishing it. Second, the answer is assembled from several sources at once, which is why AI Overviews blend multiple publishers into a single response instead of picking a winner.
What survives chunking and what does not
| How the content is written | What happens when it is chunked |
|---|---|
| Answer stated directly under a question-shaped heading | Chunk stands alone and reads as a complete answer |
| Answer split across a paragraph and a later table | One half is retrieved without the other, so the claim loses its numbers |
| Meaning carried by ‘as mentioned above’ or ‘this approach’ | The referent is in a different chunk and the passage becomes ambiguous |
| Key qualifier buried in a footnote or an aside | Retrieved text overstates the claim |
| Definition, condition, and exception in the same short section | Retrieved intact and quotable without editing |
The blunt version
Chunk-level clarity beats page-level authority, and that is the part the industry has been slowest to price in. Retrieval compares your passage against other passages. Your backlink profile is not in that comparison. A 300-word section on a modest site can be selected over a celebrated guide on a well-linked domain, because the retriever is judging the fragment in front of it.
The evidence is genuinely mixed on one point, and pretending otherwise would be dishonest. Where the retrieval layer draws candidates from a search index rather than an independent one, ranking still matters, because poor rankings keep you out of the candidate pool entirely. Authority buys the audition. It does not win the part. Once you are in the pool, selection is decided passage by passage, and that step is where most well-linked pages quietly lose.
So the practical rule inverts the usual advice. Stop asking whether the page is comprehensive. Ask whether any 200-word window of it, lifted out and read cold, still answers something and still carries its conditions. That is a self-contained editing test you can run this week, and it is the same discipline behind getting cited in AI answers. Nobody sells it as a package, because it looks like editing rather than a deliverable.
Example
Say a commercial insurance broker publishes a 3,500-word guide to public liability cover. The specific answer people ask for, what the policy excludes, sits in paragraph 41 and depends on a table two screens further down. A retriever pulls a chunk from the middle of that guide. It contains the phrase ‘the exclusions listed above’ and none of the exclusions. The passage is useless, so a thinner competitor page gets used instead. The fix is not more words or more links. It is a heading that asks the exclusion question directly, followed by the exclusions in the same short section, with the qualifying conditions written into the sentences rather than referenced elsewhere. Same guide, same domain, restructured so a fragment survives being taken out of context.
FAQ
Is RAG the same as a model being trained on my content?
No. Training happens once, in advance, and produces weights that cannot name their sources. Retrieval happens at question time and pulls live text the model can quote and link. That difference is why a page published yesterday can appear in an answer from a model whose training finished long before it existed.
Does RAG stop a model inventing things?
It reduces the problem without solving it. Grounding an answer in retrieved passages gives the model something real to work from. It can still merge two sources incorrectly, keep a claim while dropping the condition attached to it, or summarise a passage into something the source never said.
How long should a section be for retrieval?
Chunk sizes vary by system, so no single length is correct. The workable test is self-containment: a section should answer one question, carry its own qualifiers, and avoid pointing at material outside itself. If a reader could understand it with the rest of the page removed, a retriever can use it.
Related terms
- Above the Fold — the human equivalent of chunking, where position decides what gets read at all.
- Share of Voice — the metric people reach for when they try to measure AI visibility across a topic.
- AI Overview — the most visible consumer-facing product built on retrieval of this kind.
Your page does not get retrieved. A few hundred words of it do. Edit for the fragment, because that is the unit the system is actually reading.