Building with the Claude API · lesson 16 of 17

Context Management

Managing what's in the model's context window across a long agent run or conversation. A core CCA-F topic.

The problem

Claude's stateless API + 200K context + agent loops with tool calls = conversations that fill quickly. Naively appending everything yields:

Slow requests (every turn re-pays for all prior tokens unless cached).
Cost growth.
"Lost in the middle" — Claude weights ends of context over middle.
Eventually: max_tokens exhaustion or context_window_exceeded errors.

Strategies (memorize for the exam)

1. Progressive summarization

After every N turns, replace the oldest turns with an assistant summary:

[assistant] "Summary so far: User is debugging a Stripe webhook. We've confirmed the signature is valid and identified that event_type 'invoice.paid' is being dropped..."
[user] {next turn}
[assistant] {next response}

Keeps recent detail high-fidelity; older detail compressed.

2. State object pattern

Maintain a JSON state of canonical facts (cart, user info, open question). Pass via system or first user turn. Don't replay the whole conversation — the state object IS the memory.

Best for: support flows, transactional agents, workflow execution.

3. Sliding window

Keep last N turns verbatim. Drop older. Optionally combined with a global summary.

Best for: chat assistants where recency dominates relevance.

4. Retrieval-on-demand

Don't pack history into context. Store turns in a vector store. Retrieve only relevant past turns when needed.

Best for: very long-lived agents, knowledge-worker assistants.

5. Subagents (delegation)

When one task needs a deep dive, delegate to a subagent with a focused context. The parent agent only sees the subagent's summary.

Best for: research, multi-step planning, complex code work.

Stale-data hazards

Tool results expire. The "user's balance" you fetched 20 turns ago may be wrong.
When summarizing, mark when each fact was retrieved.
Prefer "refresh-on-need" over "remember-forever" for volatile facts.

Token estimation

Roughly: 1 token ≈ 4 characters ≈ 0.75 words. For careful budgeting, use the SDK's count_tokens helper. Budget categories:

Stable prefix (system + cached docs) — large but cheap on cache hits.
History — bounded.
Working memory / state object — small, dense.
Current turn — variable.

CALM framework (CCA-F mnemonic)

Cache stable prefixes.
Align prefix structure across requests (don't reshuffle).
Limit history (bound the window or summarize).
Monitor cache hits and token usage.