Project 02 — Build a Prompt Lab

Enforces: structured prompt construction and disciplined comparison (from prompt-engineering-interactive-tutorial / real-world-prompting) Surface: claude.ai (no code) · Time: ~60 min · Difficulty: 🟡 core

Why this project

Most people improve prompts by superstition — they change five things at once and credit whichever felt clever. A lab forces the opposite habit: build one strong structured prompt, then change exactly one structural element at a time and measure what moved. That's the difference between "I think examples help" and "examples cut the off-format rate from 4/10 to 0/10 on my task."

What you'll build

A prompt lab document (a claude.ai Artifact or any notes app) holding one real task, a fully-structured baseline prompt, three single-variable variants, the outputs, and a one-paragraph verdict naming the structural change that helped most.

Steps

Pick one real task — something you actually need done and can judge: extract action items from a messy meeting note, classify support emails by urgency, rewrite a paragraph in a fixed tone. Write down what a good answer looks like before you prompt.
Write the structured baseline — assemble a prompt that uses all six levers, in this order:
- Role — You are a meticulous operations analyst.
- Context — why the task exists and who the output is for.
- XML-tagged data — wrap the input so Claude can't confuse instructions with content:
```
<transcript>
...the messy text...
</transcript>
```
- Examples — one or two input → output pairs (few-shot) showing the exact shape you want.
- Output format — state it explicitly ("Return a Markdown table with columns Owner | Task | Due").
- Prefill — on the API you can begin Claude's reply for it by adding a partial assistant turn (e.g. | Owner |) to lock the format; the model continues from there. (Prefill isn't available when extended thinking is on, and for strict schemas a forced-tool output is more reliable — you'll do that in p07-structured-extraction.)
Run the baseline 3 times — same prompt, fresh chats. Note whether the output is stable or varies.
Build three single-variable variants — change exactly one element per variant so the comparison is clean:
- Variant A — strip the examples (keep everything else).
- Variant B — remove the XML tags (paste the data inline instead).
- Variant C — weaken the output-format instruction (say "summarize" instead of giving the exact schema).
Score each variant against your "good answer" definition — pass/fail on format, completeness, accuracy. Use a small grid so the comparison is visible at a glance.
Write the verdict — one paragraph: which removed element degraded the output most, and what that tells you about where the leverage is for this task.

Acceptance criteria — you're done when

Your baseline prompt visibly uses all six levers (role, context, XML-tagged data, examples, output format, prefill).
You ran the baseline at least 3 times and noted output stability.
You have three variants, each changing exactly one element from the baseline.
You scored every variant against a pre-written definition of a good answer.
Your verdict names the single structural change that mattered most and why.
You journaled the result in learning-journal-template.

Stretch goals

Add a negative example ("don't do this") and measure whether it helps or hurts.
Test the same six-lever prompt on a harder model and a faster one — does structure matter more or less per tier?
Turn your winning prompt into a reusable template with {{placeholders}} and run it on three new inputs.

Self-assessment rubric

Level	Signal
🟢 Got it	You change one variable at a time and can point to the lever that drove the result.
🟡 Almost	Your prompts are structured but your comparisons still move two things at once.
🔴 Revisit	Outputs feel random; re-read prompt-engineering-interactive-tutorial on XML tags and examples.