Answer Patterns — The Complete Lesson

The full picture

Most exam questions describe a system that is going wrong.

Your job is to name the fix. Five patterns cover almost all of them.

Pattern 1 splits big work across many agents. Pattern 2 keeps the context window small and useful. Pattern 3 fixes the shape of the output. Pattern 4 adds a human or code when a rule must hold. Pattern 5 measures if the output is good enough.

They also feed each other. Delegation protects context. Few-shot guides but never guarantees, so hard rules move to code. Code needs a threshold, and evals set that threshold.

flowchart TD
    Q["Exam question:<br/>what is going wrong?"] --> A["Too much work<br/>for one agent"]
    Q --> B["Chat is slow,<br/>costly, or forgetful"]
    Q --> C["Output format<br/>is wrong"]
    Q --> D["A rule must<br/>never break"]
    Q --> E["Is the output<br/>good enough?"]
    A --> A1["1. Multi-agent<br/>orchestration"]
    B --> B1["2. Context<br/>management"]
    C --> C1["3. Few-shot<br/>examples"]
    D --> D1["4. Human<br/>in the loop"]
    E --> E1["5. Quality<br/>and evals"]
    A1 -.->|"subagents return<br/>only summaries"| B1
    C1 -.->|"guides, never<br/>guarantees"| D1
    D1 -.->|"who gets<br/>reviewed?"| E1
    E1 -.->|"tested thresholds"| D1

1. Multi-agent orchestration

We start with the biggest design choice in the overview. One agent, or many?

The seven core rules

1. Coordinator owns the flow. It collects each subagent's output. It puts what is needed into the next subagent's prompt.
2. Structured handoffs. Subagents return structured data. That means content plus metadata. Never loose prose. Sources, dates, and IDs must not get lost.
3. Scoped tools. Give each subagent only the 4–5 tools for its job. Too many tools means wrong-tool calls.
4. Goal-based delegation. Give the goal and the quality criteria. Let the subagent pick its own strategy. Micromanaging breaks its ability to adapt.
5. Iterate on gaps. The coordinator reviews results. Then it re-delegates targeted follow-ups. Then it repeats.
6. Delegate to protect context. Heavy exploration happens inside the subagent's own context. Only the summary comes back to the coordinator.
7. Do not spawn for trivial work. If the coordinator can just answer it, let it answer. Spawning costs latency.

Parallel or sequential

flowchart TD
    A{Do tasks depend<br/>on each other?} -->|Independent| B["Parallel<br/>(multiple Task calls in ONE response)"]
    A -->|Output feeds next| C["Sequential<br/>(coordinator relays outputs)"]

For parallel work, the coordinator makes several Task calls in one response.

Spawning mechanics (Claude Code and the Agent SDK)

• Task/Agent tool. This is the tool that spawns a subagent.
• Missing "Task". If the coordinator's allowedTools list has no "Task", the exam answer is simple. The coordinator can talk about delegating. It can never actually spawn.
• Exam era vs current docs. Current SDK docs add a nuance. An unlisted tool is only "not auto-approved". To remove it fully you need disallowedTools or the tools list. Pick the exam answer above unless the question is about current SDK behaviour.
• Where custom subagents live. Put them in .claude/agents/ for the project. Put them in ~/.claude/agents/ for the user.
• Nesting: exam era vs current docs. Classic guidance says subagents do not spawn their own subagents. The coordinator spawns all of them. The newest docs allow nesting up to 3 layers. If an option says "the coordinator spawns them", pick that one.

2. Context management

Rule 6 above said delegation protects context. Now we look at context itself.

Symptom to fix

What counts against context

Everything you send in the request counts:

• the system prompt
• the tool definitions — schemas cost tokens too
• the conversation history, including every tool result
• the current document or input

The layered memory picture

flowchart TD
    A["Recent turns<br/>(verbatim — keep)"] --> D[Context window]
    B["Older turns<br/>(compressed summary)"] --> D
    C["Structured facts<br/>(scratchpad / files — persist outside)"] -.reload when needed.-> D

Traps

• "Just increase the context window" as the first fix. It costs money and time. It usually hides a design problem.
• Blind truncation from the top. You lose promises made and facts stated.
• Keeping raw 120K-token tool dumps because "we might need them".

3. Few-shot examples

Context management keeps the right data in view. Few-shot examples fix the shape of what the model writes.

When the exam wants few-shot examples

• The output format changes between runs.
• Compound phrases are handled inconsistently.
• Informal values get converted when they should stay word for word.
• Different document layouts confuse the extraction.
• Reviews cannot tell acceptable patterns from real issues.
• Any phrasing like "the model formats X wrong".

How to build them

1. Pick 2–3 real inputs. Include the failing edge cases.
2. Write the exact output you want for each one.
3. Show the full pair. Input and output, not just outputs.
4. Match the target format exactly. Same field names. Same style.

Few-shot or something else

flowchart TD
    A{Problem type} -->|Format / style / granularity| B[Few-shot examples]
    A -->|Structure must be guaranteed| C["Schema: tool input schema<br/>or structured outputs"]
    A -->|Missing info gets invented| D["Instruction: return null<br/>if not in source"]
    A -->|Behavior must NEVER happen| E[Code / hooks enforcement]

Few-shot guides. Schemas guarantee.

Be precise here. A plain input_schema still needs a validation loop. Only strict: true, or Structured Outputs, gives a hard guarantee. Read the question and know which one it asks for.

Other prompting tools (the supporting cast)

• XML tags. Tags like <instructions>, <example>, and <document> keep prompt parts apart. Use them when instructions and data mix together.
• System prompt. It holds the role and the stable rules. User turn. It holds the task and the data.
• Chain of thought. Say "think step by step" for reasoning-heavy tasks. Do not use it to fix formats.
• Too many clarifying questions? Tell the model to make reasonable assumptions, state them, and offer to adjust.

4. Human in the loop

Few-shot examples guide the model. When guiding is not enough, code or a human must step in.

The enforcement ladder

flowchart TD
    A{How strict is the rule?} -->|Preference / style| B[Prompt instruction]
    A -->|Should usually hold| C[Tool description guidance]
    A -->|Must ALWAYS hold| D["Code: hook intercepts the tool call,<br/>or the tool enforces it internally"]

Three "must always" examples the exam likes:

• Refunds over $500. A hook blocks process_refund and escalates.
• Reimbursement threshold. The tool itself checks the amount. Small amounts auto-process. Large amounts route to manager approval.
• A loop ends with no resolution. The orchestration code notices this and escalates in code.

When to escalate

See also scenarios/customer-support.mdx.

1. The customer asks for a human.
2. The action goes past the agent's authority or policy.
3. The agent is stuck and makes no real progress.

The handoff is a structured summary. Include who, what happened, the root cause, the amount, and the recommended action. Never hand over a raw transcript.

Confirmation design

• Bad: "Ready to post. Confirm?" People rubber-stamp that in 2 seconds.
• Good: show everything being approved. Full content, target account, time, and platform. The human must be able to really check it.

Review routing for extraction and quality work

• Send to a human when confidence is low. Also when sources are ambiguous or contradictory.
• Keep watching auto-approved output. Use stratified random sampling every week.
• Tune thresholds with a labeled validation set.

5. Quality, confidence and evaluation

Human review needs a rule for who gets reviewed. Confidence scores and evals give you that rule.

Confidence-driven review

This is the exam's preferred design for ML and extraction pipelines.

1. The tool or model returns field-level confidence scores.
2. Your code computes request_review using tested thresholds.
3. Include review_reasons so reviewers know why it was flagged.
4. Calibrate the thresholds against a labeled validation set.

flowchart LR
    A[Extraction +<br/>confidence scores] --> B{Code checks<br/>thresholds}
    B -->|High confidence| C[Auto-accept]
    B -->|Low confidence /<br/>ambiguous source| D[Human review<br/>+ review_reasons]
    C --> E["Weekly stratified<br/>random sample audit"]

Measurement rules

Iterating on failures

• One variable at a time. Fix, then verify, then move to the next fix.
• Bug reports become test cases. Give the model a failing case. That means the input plus the expected output.
• Validation failure. Send the error back in a follow-up request and ask for a correction.
• Re-run only the failed subset, in chunks. Never re-run the whole batch.

Recap

Multi-agent orchestration

• Subagents share nothing by default. Data moves through prompts.
• The coordinator owns the flow and feeds the next prompt.
• Handoffs are structured data, content plus metadata. Never loose prose.
• Loose prose loses sources, dates, and IDs.
• Give each subagent only 4–5 tools. Too many cause wrong-tool calls.
• Delegate goals plus quality criteria. Let the subagent pick the strategy.
• Micromanaging breaks the subagent's ability to adapt.
• The coordinator reviews results and re-delegates targeted follow-ups.
• Delegation protects context. Only the summary returns.
• Do not spawn for work the coordinator can just do. Spawning costs latency.
• Parallel for independent tasks. Use several Task calls in one response.
• Sequential when one output feeds the next step.
• Subagents spawn through the Task/Agent tool.
• No "Task" in allowedTools means the coordinator can never spawn.
• Current docs nuance: unlisted means not auto-approved. Full removal needs disallowedTools or the tools list.
• Custom subagents live in .claude/agents/ or ~/.claude/agents/.
• Classic nesting rule: subagents do not spawn subagents.
• Newest docs: nesting is allowed up to 3 layers. Prefer the classic answer on the exam.

Context management

• Long chats need progressive summarization.
• Keep the last 5–6 turns word for word.
• Trim 40+ field tool replies to the relevant fields only.
• Persist structured issue data (IDs, statuses, amounts) outside the chat.
• Use a scratchpad file for long investigations.
• Forgetting between API calls means the code drops conversation history.
• Too big for one agent means delegate to subagents.
• Extraction that decays at the end means you are near the context limit. Chunk the input.
• A stale session needs a new session, an injected summary, and fresh tool calls.
• Context holds the system prompt, tool definitions, history including every tool result, and the current input.
• Tool schemas cost tokens.
• Trap: raising the context window first. It costs money and time.
• Trap: truncating blindly from the top. You lose facts and commitments.
• Trap: keeping raw 120K-token tool dumps just in case.

Few-shot examples

• Wrong or mixed format means few-shot. Almost always.
• Use 2–3 complete input → output pairs.
• Cover the failing edge cases in your examples.
• Signals: format varies, compound phrases inconsistent, informal values wrongly converted, different layouts confuse extraction, reviews cannot separate acceptable patterns from real issues.
• Build them from real inputs, with the exact wanted output.
• Show the full pair, and match field names and style exactly.
• Format or style problem means few-shot examples.
• Guaranteed structure means a schema or Structured Outputs.
• Invented missing info means instruct the model to return null.
• Behavior that must never happen means code or hooks.
• Few-shot guides. Schemas guarantee.
• A plain input_schema still needs a validation loop.
• strict: true or Structured Outputs gives a hard guarantee.
• XML tags like <instructions>, <example>, <document> split prompt parts.
• System prompt holds role and stable rules. User turn holds task and data.
• Chain of thought helps reasoning tasks. It does not fix formats.
• For too many questions, tell the model to assume, state, and offer to adjust.

Human in the loop

• Must, never, compliance, guarantee all mean code or hooks.
• Prompts guide. They never guarantee.
• Preference or style goes in a prompt instruction.
• Should usually hold goes in the tool description.
• Must always hold goes in a hook or inside the tool itself.
• Refunds over $500: a hook blocks process_refund and escalates.
• Reimbursements: the tool checks the amount. Small auto-processes. Large goes to manager approval.
• A loop with no resolution: orchestration code escalates it.
• Escalate when the customer asks for a human.
• Escalate when the action exceeds authority or policy.
• Escalate when the agent is stuck with no real progress.
• A handoff is a structured summary: who, what happened, root cause, amount, recommended action. Not a raw transcript.
• Weak confirmation: "Ready to post. Confirm?" People rubber-stamp it.
• Strong confirmation: show full content, target account, time, platform.
• Route to a human on low confidence or ambiguous or contradictory sources.
• Audit auto-approved output with weekly stratified random sampling.
• Tune thresholds with a labeled validation set.

Quality, confidence and evaluation

• Set thresholds in code, based on tested data. The model does not set policy.
• Aggregate accuracy hides segment failures.
• Auto-approved output still needs sampled human review.
• The pipeline: field-level confidence scores → code computes request_review → add review_reasons → calibrate on a labeled validation set.
• 95% aggregate accuracy: split it by document type and by field.
• Auto-accepted high confidence: sample a fixed % weekly, stratified. It measures the error rate and catches new patterns.
• Totals: cross-check computed against stated. A mismatch goes to review.
• Unsure about a finding: report it with confidence and severity tags, then filter downstream.
• Inconsistent reviews: use focused prompts per category, each with its own examples.
• Fix one variable at a time. Fix, verify, next.
• Turn a bug report into a failing test case: input plus expected output.
• On validation failure, send the error back for correction.
• Re-run only the failed subset, in chunks. Never the whole batch.