Retrieval-augmented generation

Retrieval-augmented generation first finds relevant source passages and then asks a generative model to answer with that context. Retrieval improves grounding, but poor chunks, search, permissions, or evaluation still produce unreliable answers.

At a glance

Question	Practical answer
When is it useful?	A policy assistant searches approved handbook sections, supplies the top passages to an LLM, and returns an answer with links and a fallback when evidence is weak.
What should you do?	Create ten short documents and five questions, compare answers with and without retrieval, and inspect whether the correct passage reached the prompt.
How do you know it worked?	Each answer cites a supporting passage, permission boundaries hold, unsupported questions abstain, and a fixed evaluation set tracks regressions.
Common failure	Adding more retrieved text can reduce quality; optimize relevance, chunk boundaries, metadata filters, and context budget together.

flowchart LR
  A[Question] --> B[Retrieval-augmented generation]
  B --> C[Small example]
  C --> D[Evidence]

The important idea is not to stop at a definition: connect the concept to a small example and observable evidence.

Worked example

A policy assistant searches approved handbook sections, supplies the top passages to an LLM, and returns an answer with links and a fallback when evidence is weak.

Before acting, write the success signal. Change one condition at a time, observe the result, and record assumptions. For Retrieval-augmented generation, this separates what you know from what you are merely guessing.

Practice in 20–30 minutes

Goal: Create ten short documents and five questions, compare answers with and without retrieval, and inspect whether the correct passage reached the prompt.

1. Record the starting state and your prediction.
2. Implement the smallest version without adding unnecessary tools.
3. Change exactly one input or constraint and repeat.
4. Save a command, screenshot, output, or checklist as evidence.

Expected result: Each answer cites a supporting passage, permission boundaries hold, unsupported questions abstain, and a fixed evaluation set tracks regressions.

What can go wrong

Adding more retrieved text can reduce quality; optimize relevance, chunk boundaries, metadata filters, and context budget together.

When the result differs from your prediction, do not change many things at once. Check inputs, versions, environment, permissions, and logs, then repeat from the smallest example.

Definition of done

• I can explain the concept in my own words.
• I completed the small example and kept evidence.
• I know one failure mode and how to check it.
• Someone else can repeat the work without guessing missing steps.

Go deeper

Use the linked resource or repository at the end of the page when you need a full implementation. Check current versions before applying commands to a real project.