RAG Patterns with Claude

Retrieval-Augmented Generation: pull relevant chunks from a corpus, drop them into context, let Claude answer grounded in those chunks.

The basic loop

1. Embed query.
2. Vector-search corpus, retrieve top-K chunks.
3. Insert chunks into user/system content with citation markers.
4. Instruct Claude: "Answer only from <context>. If absent, say so. Cite sources."
5. Generate.

Long-context vs RAG

Claude's 200K+ context tempts you to skip retrieval. Tradeoffs:

Approach	Pros	Cons
Dump entire corpus	Simple. No retrieval infra.	Expensive per call. Slower. "Lost in the middle" — Claude weights edges over middle.
Classic RAG (top-K chunks)	Cheap, fast, traceable.	Retrieval quality is the bottleneck.
Hybrid (recommended)	Cache the long doc; retrieve which sections to focus on.	Slightly more complex.

Contextual retrieval

An Anthropic-recommended technique: when chunking, prepend each chunk with a few-sentence contextual prefix describing where the chunk sits in the broader document. Retrieval quality jumps 30–50%.

[Chunk context: This excerpt is from section 4 of the 2025 ACME annual report, discussing supply chain risks.]
[Original chunk text here.]

The context is generated once at indexing time by a cheap Claude call (Haiku, cached system prompt).

Citation pattern

Ask Claude to emit source markers inline:

Wrap quoted material in <quote source="doc_id">...</quote>.
After your answer, list all source IDs used.

Or use a tool call schema that requires citations as fields.

Grounding instructions

System prompt should include:

You are answering from <context> only.
- If the context does not contain the answer, say: "I can't find that in the provided context."
- Do not use prior knowledge.
- Quote exact phrases when stating specific facts.

This dramatically reduces hallucinations.

Caching the corpus

For a stable knowledge base + variable user query:

system=[
    {"type": "text", "text": persona},
    {"type": "text", "text": LARGE_KB_CHUNKS, "cache_control": {"type": "ephemeral"}},
]
messages=[{"role": "user", "content": query}]

Refresh cache when KB updates. Inside the 5-minute (or 1-hour) window every query is cheap.