Explained simply · Free to read · 4 min read

RAG explained simply: how AI answers from your own documents

RAG stands for retrieval-augmented generation, roughly “answering after looking things up”. Before each answer, the AI searches a defined collection, such as your PDFs or a company handbook, pulls out the relevant passages and bases its answer on them. That lets it answer questions about knowledge it never saw in training, without the model having to be retrained. You already use RAG in Gemini Notebook (formerly NotebookLM), in ChatGPT projects with uploaded files and in many company chatbots. It does not eliminate errors completely, so it pays to check the sources it shows (as of October 2026).

Published 11 October 2026

In short

RAG (retrieval-augmented generation): RAG is a technique in which an AI looks things up in a defined collection of documents before answering and bases its answer on the passages it finds, without the language model having to be retrained.^1,5,6; Example: You upload the manual for your heating thermostat to Gemini Notebook and ask how to set the night-time setback. The AI finds the right page, explains the steps and shows you where in the manual it says so.; Also called: retrieval-augmented generation · AI with your own documents · chat with PDF

What RAG means

The abbreviation combines three words: retrieval, augmented and generation. The idea is that before the AI answers, it retrieves relevant information from an external knowledge collection and bases its answer on it.

OpenAI explains the difference with an exam. Either you have learned the material in class over months, or you are allowed to bring the textbook and look things up. RAG is the open textbook: the model does not have to carry the knowledge in its head, it is handed the right pages at the moment of the question.

The term comes from a research paper by Patrick Lewis and colleagues, published in May 2020 and accepted at the NeurIPS 2020 conference. Back then, Wikipedia served as the reference work. Today, cloud providers such as AWS and Google describe RAG as a way to connect language models to a company's own or up-to-date knowledge.

How it works in three steps

First, you ask a question, for example “How long is the warranty on my device?”. Second, a search component looks through the stored documents and pulls out the passages that best match the question. Third, those passages are handed to the language model together with your question, and it writes an answer in full sentences, often with a pointer to where it found the information.

The big advantage, according to AWS: the language model itself does not need retraining. When the knowledge changes, you simply swap the documents. That is cheaper and faster than retraining, which experts call fine-tuning. We explain the difference between the two in a separate glossary entry.

Worth knowing: RAG is not a single program but a design principle. It sits inside many services without the word ever appearing.

Where you already use RAG

Gemini Notebook: Google's research tool was called NotebookLM until July 2026 and, according to Google, is the same product under a new name. You upload your own sources, such as PDFs, texts or links, and Gemini Notebook answers questions based on those sources. If the answer is not in them, Google says it cannot answer the question. Free use covers 100 notebooks with up to 50 sources each and 50 chat queries per day; one source can contain up to 500,000 words, and an uploaded file can additionally be no larger than 200 MB. With Google AI Plus you get 100 sources per notebook, with AI Pro 300 and with AI Ultra up to 600. According to Google, it is available wherever the Gemini app is offered, which includes Germany. You need to meet the minimum age for a Google Account, which in Germany is 16 for a personal account; school and work accounts have their own rules.

ChatGPT projects: in a project you can store PDFs, spreadsheets, documents or images that ChatGPT uses for its answers. According to OpenAI, the free plan allows 5 files per project, Go and Plus 25, and Pro, Business, Edu and Enterprise 40.

Company chatbots: when the chat on your insurer's website or the help tool on your employer's intranet quotes from internal handbooks, this principle is usually behind it. AWS describes exactly this as the purpose: connecting language models to an organisation's internal knowledge base without retraining them.

Limits: what RAG does not solve

RAG lowers the risk of invented answers but does not remove it. OpenAI describes two typical sources of error: the search returns the wrong passages, or so much irrelevant material that the real information gets lost. Or the model gets the right passages and still does the wrong thing with them. For important questions, check the passage it points to. According to Google, Gemini Notebook shows source references, although for very short sources it only refers to the whole document.

The documents themselves can also be a way in. The OWASP security project describes a case where someone tampers with a document in the knowledge collection and hides instructions to the AI inside it. In 2026, Germany's Federal Office for Information Security (BSI) published a paper on basic protection against such attacks on document-based AI chats. Our glossary entry on prompt injection goes into more detail.

And: everything you upload goes to the provider. For personal accounts, Google says content from Gemini Notebook is not used for training unless you give feedback on an answer. Even so, other people's confidential documents do not belong in a cloud service.

Sources

Created with AI assistance and editorially approved before publication.