Vault / wiki/301/practice/domain-4-prompting.md
updated 2026-05-28Practice — Domain 4: Prompt Engineering & Structured Output (20%)
18 scenario-based MCQs. Answer key + explanations at the bottom.
Q1
Your structured-output JSON is inconsistent — sometimes missing fields, sometimes the wrong types. Which is the most reliable fix?
A. Add stricter wording in the system prompt asking for valid JSON
B. Switch to forced tool use with tool_choice = {type: "tool", name} and the JSON schema in input_schema
C. Use prefill with {
D. Switch to a bigger model
Q2
Your system prompt is 4,000 tokens with many conditional rules. One rule ("never quote a price without a date") keeps getting violated. Most likely root cause?
A. The model can't read more than 2,000 tokens B. Dilution — the rule's weight drops among many sibling rules C. Caching is interfering with rule application D. The rule should be in the user turn
Q3
For best results when adding examples to a prompt, how many examples are typically the sweet spot?
A. 1 B. 3–5 C. 10–15 D. 20+
Q4
You're extracting invoice fields and must support audit. Which schema design enables auditability?
A. Each field paired with a <field>_source pointer to where in the document it came from
B. Use additionalProperties: false
C. Mark all fields required: true
D. Wrap the schema in an enum
Q5
You're bulk-classifying 500K customer messages overnight. Cost matters; latency does not. Best combination?
A. Streaming Sonnet with prompt caching B. Message Batches API + Haiku + forced tool use + cached system prompt with few-shot examples C. Synchronous Sonnet in a single long-running script D. Orchestrator–workers with one subagent per message
Q6
A validation loop should:
A. Loop indefinitely until valid B. Validate generated output, on failure re-prompt the model including the validator's error, cap retries (~3), route to human review if still failing C. Skip validation if the model is large enough D. Validate only on the first attempt
Q7
"Principles vs conditionals" — which approach is generally more robust under prompt dilution?
A. Principles ("always cite sources") B. Conditionals ("if user is angry, then escalate") C. They're equivalent D. Neither — only few-shot examples are reliable
Q8
Which prompting technique is not in Anthropic's canonical "in order" list of seven?
A. Be clear and direct B. Use XML tags C. Use embeddings D. Prefill Claude's response
Q9
When picking few-shot examples for a classification task, you should prioritize:
A. Three examples of the same class (the most common) B. Diverse examples covering typical, ambiguous, and edge-case classes C. Long, verbose examples D. Examples in a different output format than what you want
Q10
For an interactive UI assistant where time-to-first-token matters, you should:
A. Use the Message Batches API
B. Use streaming via client.messages.stream
C. Disable extended thinking always
D. Avoid tools
Q11
Extended thinking is most appropriate for:
A. High-volume routing on Haiku B. Multi-step reasoning tasks where a one-shot answer is wrong C. Pure classification tasks D. Streaming UIs with tight first-token SLAs
Q12
You force a tool with tool_choice = {type: "tool", name: "extract"}. Where do you find the structured result in the response?
A. Inside the first text content block as JSON
B. Inside the tool_use block's input field as a dict matching input_schema
C. As an HTTP header
D. The next API call
Q13
A system prompt has 50KB of stable reference material plus a small per-request task. To make this cheap and fast on repeated calls, you should:
A. Include the reference inline in every user turn
B. Put it as a system-prompt content block with cache_control: {type: "ephemeral"}
C. Switch to Haiku
D. Compress with gzip
Q14
A few-shot block should be wrapped in:
A. Triple backticks
B. XML tags (e.g., <examples><example>...</example></examples>)
C. JSON arrays
D. Curly braces
Q15
Why prefill the assistant turn with {?
A. To shorten the prompt B. To constrain Claude to immediately produce JSON, reducing chatter and improving structural compliance C. To bypass the system prompt D. To force tool use
Q16
The CCA-F exam favors which structured-output approach for production extraction pipelines?
A. Prompt-only "respond in JSON" B. Forced tool use with a JSON schema, validation loop, and capped retries to human review C. Manual regex post-processing D. Asking Claude to re-format its prose answer
Q17
A team writes a 500-line system prompt that has grown over months. They add a new "never approve refunds over $X" rule, and now older rules start being missed. Best next step?
A. Add more rules to compensate
B. Split into focused subagents and a routing layer; move stable reference into cached blocks
C. Switch model
D. Increase max_tokens
Q18
When you batch-process 10K extraction tasks, prompt caching:
A. Doesn't apply to batches B. Applies — the cached system prompt across all requests in the batch lowers cost dramatically C. Increases batch cost D. Disables forced tool use
Answers
Q1: B. Forced tool use is the most reliable structured-output technique. Prefill (C) is second. Prompt-only (A) is the least reliable. Bigger model (D) without changing the technique still produces drift.
Q2: B. Classic dilution. Mitigations: move critical rules to top and end, split into focused subagents, move stable reference into cached blocks.
Q3: B. Anthropic recommends 3–5 examples as the sweet spot — enough to teach the format and edge cases, not so many that they bloat context or overfit.
Q4: A. Audit requires provenance — each value linked to its source location. (B)/(C)/(D) are unrelated to auditability.
Q5: B. Bulk + offline → Batch API (50% off). Haiku is cost-aligned for high-volume classification. Forced tool use is the reliable output channel. Cached system prompt with few-shot is the canonical setup.
Q6: B. Validate → retry-with-error-feedback → cap retries → human review on persistent failure. Infinite loops (A) burn cost; skipping validation (C)/(D) lets bad data through.
Q7: A. Principles ("always X") survive dilution because they apply universally. Conditionals accumulate and break — prefer principles where possible.
Q8: C. Embeddings aren't on Anthropic's canonical list of seven. The seven: clear/direct, examples, let Claude think, XML tags, role, prefill, chain.
Q9: B. Diverse examples covering typical, ambiguous, and edge cases teach more than three near-duplicates. Format mismatches (D) drift outputs.
Q10: B. Streaming is the way to optimize time-to-first-token. Batches (A) are async; disabling thinking (C) is sometimes right but unrelated to TTFT in the streaming sense; avoiding tools (D) is unrelated.
Q11: B. Extended thinking is for hard multi-step reasoning where one-shot fails. Cheap routing (A) wastes thinking budget; streaming (D) doesn't pair well.
Q12: B. Forced tool use returns the structured data as the input of the tool_use block. It's already a dict — no JSON parsing needed.
Q13: B. Cache the stable prefix. Inline-every-turn (A) destroys cost economics. Switching models (C) doesn't help with this.
Q14: B. XML tags. Wraps examples so Claude doesn't confuse them with the actual task.
Q15: B. Prefilling { constrains Claude's continuation to JSON. Reduces preamble and structural drift.
Q16: B. Forced tool use + validation loop + capped retries + human review is the canonical extraction architecture.
Q17: B. When conditionals proliferate, the right structural move is to split into focused subagents with a routing layer in front. (A) makes dilution worse.
Q18: B. Prompt caching applies inside the Batches API as well — cached system prompts amortize across the requests in the batch for massive savings.