Domain 4 — Prompt Engineering & Structured Output (20%)

How to write prompts that work and how to get reliable structured data out.

Topics

4.1 The five (+ two) prompt levers

1. Be clear and direct.
2. Few-shot examples (2–5).
3. Let Claude think (CoT or extended thinking).
4. XML tags for structure.
5. Assign a role.
6. Prefill assistant turn.
7. Chain prompts.

Apply in order; only escalate when needed.

4.2 System prompt structure

• Identity / role
• Mission
• Tone / style
• Capabilities & tools
• Boundaries / NEVERs
• Output format
• Few-shot examples

Critical rules at top AND end of system prompt (recency + primacy).

4.3 Principles vs conditionals

• Principles ("Always cite sources") survive dilution.
• Conditionals ("If user mentions X, do Y") accumulate and break.
• Prefer principles; minimize conditionals; split into subagents when conditionals proliferate.

4.4 Dilution

• Long prompts with many same-priority rules cause Claude to weight poorly.
• Mitigations: move criticals to ends; cache stable doc material; split via subagents.

4.5 Few-shot

• 3–5 examples in XML tags.
• Cover edges: typical, ambiguous, refusal.
• Format examples identically to desired output (down to whitespace).
• Diversity > similarity.

4.6 Structured output — three techniques (ranked)

1. Force tool use with tool_choice = {type: "tool", name} — most reliable.
2. Prefill assistant turn with { or <output>.
3. Prompt-only ("respond in JSON matching...") — least reliable.

4.7 JSON schemas

• Use enum for fixed sets.
• Mark nullable explicitly.
• Minimize required to truly required.
• Add description to every property.
• Flatten when possible; deep nesting → mistakes.

4.8 Validation loops

• Always validate after generation (jsonschema, pydantic).
• On failure: re-prompt with validator error included in user turn.
• Cap retries (~3) → route to human.

4.9 Provenance for extraction

• Pair every extracted value with a source pointer.
• Schemas can require it (vendor_source etc.).
• Enables audit + spot-check.

4.10 Batch processing

• Use messages.batches API for bulk extraction.
• 50% input + output discount.
• Async, up to 24h SLA.
• Per-request custom_id to map back.
• Mix models in one batch for eval.

4.11 Evaluations

• Always have an eval set before tweaking prompts.
• Compare two prompt versions side-by-side on the same eval.
• Track regressions; a "better" change for one input class may hurt another.

Common question shapes

• "JSON output is inconsistent. What's the most reliable fix?" → Force tool use.
• "Prompt is long; one rule keeps being ignored. Why?" → Dilution; move rule to ends or restructure.
• "Bulk classify 200K rows. Best approach?" → Message Batches + Haiku + few-shot in cached system prompt.
• "Extraction must be auditable." → Include provenance pointers in schema.
• "Validation fails 10% of the time." → Validation loop with retry + cap + human escalation.

Cheatsheet

• Force structure via tools, not prose.
• Few-shot for tone/format.
• Principles > conditionals.
• Validate after generate, retry with error.
• Batch for bulk offline.

See also

• prompt-engineering-basics
• system-prompts
• few-shot-prompting
• structured-output
• tool-use
• batch-api