Privacy Notice: When you sign in with GitHub, your quiz attempts are stored in Google Firebase so you can track your progress. This data is only accessible to the course instructor, is not otherwise shared, and will be deleted at the end of the semester. For earlier deletion, contact the instructor. Prefer to stay anonymous? Use guest mode or a GitHub account without your real name.
Loading...
Sign in to track your progress and see your results across devices.
●Progress is being saved
06 Prompt Strategy Diagnosis: 12 Cards
Score:0 / 36
Question Overview
PendingCorrectIncorrectSubmitted
Goal
In this quiz you treat a dozen real-world software-development prompts as objects of study. For each card you answer two graded questions — first the examples axis (zero-shot vs few-shot), then the additive techniques present — and then read the instructor-key reveal and write a free-text reflection. By the end you will:
Predict failure modes mechanistically rather than after-the-fact;
Separate influence (what the prompt biases) from enforcement (tests, schemas, CI, review) — the three-layer model from chapter 8;
Improve weak prompts surgically rather than by making them longer.
Lecture 7 will revisit the agentic and tool-grounded cards in the orchestration-layer story.
Learning objectives
After working through this quiz, you should be able to:
Classify prompt strategies used in software-development tasks.
Identify evidence for that classification in the prompt text.
Explain why a prompt strategy may be appropriate or inappropriate for a given task.
Recognize likely failure modes, including factual hallucination, structural hallucination, overconfident explanations, format drift, missing context, and false confidence.
Separate influence from enforcement by deciding what can be handled through prompting and what requires tools, tests, schemas, CI, review, or other deterministic checks.
Improve prompts by adding missing context, examples, constraints, output formats, validation steps, or tool instructions.
Avoid prompt folklore by justifying prompt design decisions mechanistically and practically.
influence layer vs. enforcement layer vs. human layer (chapter 8)
validation through tests, schemas, linters, type checkers, CI, and code review
Two axes, not one menu
The single most important idea in this quiz is that examples are a separate question from everything else. “Did this prompt use worked examples?” is a binary axis — every prompt is either zero-shot or few-shot, and a prompt can be few-shot and also use a role, a rubric, structured output, and so on. The older single-checkbox design entangled these: it forced you to decide whether “zero-shot” belonged in the same list as “structured-output,” which has no clean answer, so the key silently used an unstated rule and the same evidence was marked correct on one card and wrong on another.
The fix is to ask two questions per card:
Q-A (the examples axis): zero-shot or few-shot? A single binary choice with a printed rule — see below.
Q-B (the additive techniques): which of the six additive techniques (role, rubric, structured-output, CoT, context grounding, tool-augmented) are present? Zero-shot/few-shot is not an option here, and neither is validation/enforcement.
This way every graded answer is uniquely derivable from a printed rule, and “few-shot” never competes with “structured-output” for a slot.
How this quiz’s categories map to the lecture’s pattern library
Chapter 16 names six patterns. This quiz uses a broader taxonomy that includes orchestration-adjacent categories (tool-augmented, structured-output) and the “outside the prompt” layer. The mapping:
Lecture pattern (chapter 16)
Quiz category
Zero-shot
Examples axis (Q-A)
Temperature
not exercised here — see "Temperature Experiments" exercise
Few-shot
Examples axis (Q-A)
Chain-of-thought
Chain-of-thought / planning (Q-B)
Self-consistency
not exercised here — orchestration concern, Lecture 7
Role priming
Role priming (Q-B)
The additional additive techniques appear in the reference sheet below because real software-development prompts compose them with the lecture’s patterns:
Rubric-based prompting — explicit evaluation criteria layered on top of role priming or structured-output prompting.
Structured-output prompting — the influence-layer version of constrained decoding (§15.1). Prompting for JSON is not the same as enforcing JSON.
Context grounding — directly applies the attention-competition point from Lecture 5: relevant context concentrates the model’s attention on what matters, while irrelevant context adds distractors that can pull the next-token decision off course.
Tool-augmented prompting — moves work out of the prompt and into the orchestration layer (Lecture 7).
Prompt strategy reference sheet
Use this as your cheat sheet while answering. The examples axis (zero-shot / few-shot) is the binary question Q-A; the six additive techniques are the Q-B menu. The “How to decide this box” column is the printed rule that makes every graded answer uniquely derivable.
Strategy
Axis or technique
Short description
How to decide this box
Main risk
Zero-shot vs Few-shot
Examples axis (Q-A, binary)
Whether the prompt demonstrates the desired behaviour with worked examples
Few-shot iff the prompt contains ≥1 worked example demonstrating the desired input→output behaviour or style to imitate — including an example built on a sibling artifact (a different function/endpoint/record). The single artifact the task operates ON (function-under-test, report-to-classify, diff-to-review) is NOT a demonstration. Otherwise zero-shot.
Zero-shot: generic/underspecified output. Few-shot: examples may bias wrongly or overfit
Role priming
Additive technique (Q-B)
Assign a professional role or perspective
An "Act as a <role>" / "You are a <role>" expertise persona. "You are helping in our X project" is membership, not a role.
Role does not guarantee expertise or correctness
Rubric-based prompting
Additive technique (Q-B)
Provide evaluation criteria/checklist
An explicit list of evaluation criteria / scoring dimensions to judge against — not a list that merely organises output sections.
May become checklist-like and miss context
Structured-output prompting
Additive technique (Q-B)
Request a fixed output format (table, JSON, sections)
A concrete output-format spec — a named serialization (JSON/YAML/table), an enum of allowed values, an "only output X" shape, OR a section schema whose parts are explicitly named/enumerated as a labeled list. Two prose imperatives joined by "and" ("explain the cause AND propose a fix") are a task description, NOT structured-output.
Format is influenced, not guaranteed (§15.1)
Chain-of-thought / planning
Additive technique (Q-B)
Ask for stepwise reasoning, plan, or rationale
Explicit "think step by step" / "show your reasoning" / "first plan, then…". A bare "explain" does NOT count.
Visible reasoning may be verbose or unfaithful (§13.5)
Context grounding
Additive technique (Q-B)
Provide relevant project/code/docs/context
Embedded background the model conditions on while producing output about something else (project conventions, related/example code, docs, a diff, a stack trace + surrounding code). The artifact that IS the task's direct object does not count.
Irrelevant or stale context can mislead
Tool-augmented prompting
Additive technique (Q-B)
Instruct model to use tools, docs, tests, APIs
The prompt names a tool/resource to invoke ("use the web search tool", "run the test suite"). Merely implying tool use ("commit them") does not count.
Tool outputs must be checked and scoped (Lecture 7)
Note — validation/enforcement is not on the graded menu. External checks (schema validation, tests, CI, review) are chapter 8’s enforcement layer — the thing prompting cannot do. It is not a prompting strategy, so it has been removed from the Q-B menu. It lives only in the reflection’s “what does the task NEED outside the prompt?” prompt.
Prompting belongs to the influence layer. Tests, schemas, static analysis, CI, sandboxing, and review belong to the enforcement and human layers.
How the quiz is structured
Each card produces two graded questions plus one ungraded reflection:
Examples axis (multiple-choice). Zero-shot or few-shot? Graded; immediate feedback on submit, and the key reveals.
Additive techniques (multiple-select). Which of the six additive techniques are present? Graded by exact-set match.
Reflection (free-text, ungraded). Write out the primary lever, the textual evidence, why the strategy fits or doesn’t, the failure modes, what belongs outside the prompt, and your improved rewrite. After you click Submit & reveal reference, the reference answer for each field reveals so you can compare. Your text stays in your browser (localStorage) only; nothing is sent to Firebase. This is the part where the actual learning happens.
That means the deck has 12 cards = 24 graded questions + 12 ungraded reflections = 36 total. The score on the progress bar reflects the 24 graded questions only.
A note on “anti-pattern” cards
Cards 5, 7, 10, 11, and 12 are deliberately weak prompts. The anti-pattern judgment lives in the reflection — specifically in what the task NEEDS that the prompt does not provide — never in the graded checkboxes. The graded classification of a weak zero-shot prompt is still zero-shot on the axis, plus whatever additive techniques (often None-of-these) are actually present. What makes a card an anti-pattern is whether that classification is the right tool for the task, which is an appropriateness judgment you record in the reflection. Reveal the explanation and write the reflection to learn why the strategy is the wrong fit.
Where to go for the group-workshop format
This page is the interactive self-study version. If you want to run the same twelve cards as a group workshop in class — with the instructor warm-up, group activity options, rubric, grading scale, and debrief questions — the companion exercise page is the instructor and group-workshop guide.
Final takeaway. Prompting is influence. Software engineering turns influence into reliable systems.
Question1
Code Generation
Card 1: Simple code generation (normalize_email) — a contrastive triple
Task: Generate a helper function for a Python project. Here is the same task written three ways. Read all three, then answer this question and the next.
Version 1a — bare:
Write a Python function `normalize_email` that strips whitespace, lowercases,
and raises ValueError unless there is exactly one @.
Version 1b — with worked examples:
Write a Python function `normalize_email` that strips whitespace, lowercases,
and raises ValueError unless there is exactly one @.
Examples:
normalize_email(" Foo@Bar.COM ") -> "foo@bar.com"
normalize_email("a@@b.com") -> ValueError
Version 1c — with an output constraint:
Write a Python function `normalize_email` that strips whitespace, lowercases,
and raises ValueError unless there is exactly one @.
Only output the function. No prose.
Three versions of the SAME task are shown. Which is the FEW-SHOT version (contains worked example calls)?
✓Correct!
Examples axis. Few-shot iff the prompt contains at least one worked example demonstrating the desired input→output behaviour. Version 1b is the only one that does: the two example calls (normalize_email(" Foo@Bar.COM ") -> "foo@bar.com" and normalize_email("a@@b.com") -> ValueError) demonstrate the exact behaviour to imitate. Versions 1a and 1c carry no demonstration, so they are zero-shot.
✗Not quite.
Hint 1
This axis is about whether any version shows the model concrete input→output pairs to imitate, not about the behaviour each version describes. Compare the three blocks for a worked demonstration.
Hint 2
Few-shot needs a worked input→output demonstration. Two of these blocks only describe the behaviour in prose; one adds something that shows it on concrete cases — look at what was appended below the shared instruction.
💡Solution
Examples axis. Few-shot iff the prompt contains at least one worked example demonstrating the desired input→output behaviour. Version 1b is the only one that does: the two example calls (normalize_email(" Foo@Bar.COM ") -> "foo@bar.com" and normalize_email("a@@b.com") -> ValueError) demonstrate the exact behaviour to imitate. Versions 1a and 1c carry no demonstration, so they are zero-shot.
Question2
Code Generation
Card 1 continued — same three versions of normalize_email as above.
Which version adds a STRUCTURED-OUTPUT constraint?
✓Correct!
Structured-output trigger. A structured-output constraint is a concrete spec for the shape of the output — here an “only output X” instruction. Version 1c says “Only output the function. No prose,” which constrains the output shape. Versions 1a and 1b say nothing about the output format. (Note that the example calls in 1b are demonstrations on the examples axis — they are not an output-format spec.)
✗Not quite.
Hint 1
Is this about whether a version constrains the shape of what comes back, or about the behaviour the function should have? Ask which block says something about the response itself.
Hint 2
A structured-output spec is a concrete constraint on output shape — an “only output X” instruction counts. Be careful: the worked example calls in one version are demonstrations on the examples axis, not an output-format spec.
💡Solution
Structured-output trigger. A structured-output constraint is a concrete spec for the shape of the output — here an “only output X” instruction. Version 1c says “Only output the function. No prose,” which constrains the output shape. Versions 1a and 1b say nothing about the output format. (Note that the example calls in 1b are demonstrations on the examples axis — they are not an output-format spec.)
Question3
Code Generation
Card 1 reflection — the normalize_email triple
Now write out your full diagnosis for Card 1 in plain prose. There is no automatic grading — after you submit, the reference answer for each field reveals so you can compare.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
For all three versions the dominant lever is the bare instruction itself: the precise, enumerated behaviour spec (“strip whitespace, lowercase, raise ValueError unless exactly one @”). That is what actually pins down a correct implementation. The worked examples in 1b sharpen the edge cases (uppercase, double @) and the “only output the function” constraint in 1c tidies the shape — but neither would save a vague instruction. Specify the behaviour first; add examples and shape second.
☰Reference answer
1a (bare): the instruction alone — strip, lowercase, raise on != one @. Zero-shot, no output constraint.
1b (few-shot): adds two worked example calls that demonstrate the desired behaviour on edge cases (uppercase normalisation, the double-@ error). This is what flips the examples axis to few-shot.
1c (spec’d): adds “Only output the function. No prose,” a structured-output constraint on the shape of the response. Still zero-shot.
☰Reference answer
The task is small, familiar, and low-risk. The behaviour is explicitly specified, so even bare 1a is probably enough to get a plausible implementation. The examples in 1b mostly help the model lock the edge-case behaviour; the constraint in 1c mostly helps a downstream consumer that wants just the function.
☰Reference answer
The model may implement a simplistic email check and overinterpret the validation requirement.
It may include extra explanation despite “Only output.”
It may not handle non-string inputs unless specified.
It may pass obvious examples but fail edge cases.
☰Reference answer
Unit tests for whitespace, uppercase, missing @, multiple @, empty string.
Static analysis or type checking if the project uses type hints.
None of the three versions includes tests; the examples in 1b are demonstrations for the model, not an executable test suite.
☰Reference answer
Write a Python function called `normalize_email(email: str) -> str`.
Requirements:
- Strip leading and trailing whitespace.
- Convert the result to lowercase.
- Raise `ValueError("invalid email")` if the normalized string contains anything other than exactly one `@`.
- Do not perform full RFC email validation.
- Return the normalized email address.
Only output the function body with the function definition. No explanation.
Task: Generate code that follows an existing project style.
Prompt:
You are helping in our Python backend project.
Implement a function that loads a user by ID and returns `None` if the user is not found.
Use our project style:
Example 1:
```python
defget_order_by_id(db:Session,order_id:UUID)->Order|None:result=db.execute(select(Order).where(Order.id==order_id))returnresult.scalar_one_or_none()```
Example 2:
```python
defget_invoice_by_id(db:Session,invoice_id:UUID)->Invoice|None:result=db.execute(select(Invoice).where(Invoice.id==invoice_id))returnresult.scalar_one_or_none()```
Now write:
```python
defget_user_by_id(db:Session,user_id:UUID)->User|None:...```
Only output the completed function.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: few-shot. The prompt contains two worked demonstrations (get_order_by_id, get_invoice_by_id) built on sibling artifacts — different functions from the one being written. Per the examples-axis rule, an example built on a sibling artifact still counts as a demonstration. The target get_user_by_id skeleton is the artifact the task operates on, not a demonstration.
✗Not quite.
Hint 1
This axis is about whether worked demonstrations are present that show the desired input→output style to imitate. Look at get_order_by_id and get_invoice_by_id versus the get_user_by_id skeleton — which role does each play?
Hint 2
Few-shot needs a worked input→output demonstration — and a sibling-artifact example (a different function from the one being written) still counts. The target skeleton is the artifact the task operates on, not a demonstration.
💡Solution
Examples axis: few-shot. The prompt contains two worked demonstrations (get_order_by_id, get_invoice_by_id) built on sibling artifacts — different functions from the one being written. Per the examples-axis rule, an example built on a sibling artifact still counts as a demonstration. The target get_user_by_id skeleton is the artifact the task operates on, not a demonstration.
Question5
Code Generation
Card 2 continued — same get_user_by_id prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Structured-output fires: “Only output the completed function” constrains the output shape to a code completion.
Context grounding fires: the two example functions embed the project’s naming, return-type, and SQLAlchemy conventions that the model conditions on while writing a different function.
Not role priming: “You are helping in our Python backend project” is project membership, not an “Act as a " expertise persona.
Double-count disclosure. The same Example 1 / Example 2 block legitimately fires both the few-shot axis (Q-A) and the context-grounding box here: as worked demonstrations they set the examples axis, and as embedded project conventions the model conditions on they ground the output. The phrase “our Python backend project” alone is only membership — it is the example code that does the grounding.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: what constrains the output shape, and what embedded conventions does the model condition on while writing a different function?
Hint 2
Apply each rule in turn. Structured-output fires on an “only output X” shape spec. Context grounding fires on embedded background the model conditions on — and remember “helping in our project” is membership, not a role; the example code is what carries the conventions.
Hint 3
Two distractors are tempting: there is no “Act as a <role>” persona, and nothing asks the model to think step by step. Recheck the role and chain-of-thought rules before you include them.
💡Solution
Structured-output fires: “Only output the completed function” constrains the output shape to a code completion.
Context grounding fires: the two example functions embed the project’s naming, return-type, and SQLAlchemy conventions that the model conditions on while writing a different function.
Not role priming: “You are helping in our Python backend project” is project membership, not an “Act as a " expertise persona.
Double-count disclosure. The same Example 1 / Example 2 block legitimately fires both the few-shot axis (Q-A) and the context-grounding box here: as worked demonstrations they set the examples axis, and as embedded project conventions the model conditions on they ground the output. The phrase “our Python backend project” alone is only membership — it is the example code that does the grounding.
Question6
Code Generation
Card 2 reflection — get_user_by_id
Now write out your full diagnosis for Card 2 in plain prose. There is no automatic grading — after you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The worked examples (few-shot) are the primary lever. The challenge here is not general Python knowledge but matching local conventions, and the examples carry naming, SQLAlchemy usage, return style, and type hints all at once. Remove them and the model falls back to generic ORM code that may not match the project; remove the “only output” constraint and you lose nothing but a little tidiness.
☰Reference answer
Two demonstration examples are provided (get_order_by_id, get_invoice_by_id).
The examples show the desired pattern (naming, SQLAlchemy usage, return type).
The target function skeleton follows the same structure as the examples.
“Only output the completed function” constrains the output to a code completion (structured-output).
The example functions embed the project’s conventions the model conditions on (context grounding). The same block fires both the few-shot axis and the context-grounding box; “our Python backend project” alone is membership, not the grounding trigger.
☰Reference answer
The main challenge is not general Python knowledge but matching local project conventions. Few-shot examples are useful here because they show naming, SQLAlchemy usage, return style, and type hints all at once — much more compactly than a prose description would.
☰Reference answer
The model may assume imports exist that don’t.
It may hallucinate the User model or its table name.
It may not know whether User.id is the correct primary-key field.
It may reproduce a pattern that is locally consistent but architecturally wrong.
☰Reference answer
Type checking.
Unit or integration test using a test database.
Code review to confirm the query belongs in this layer.
☰Reference answer
Add the relevant imports or surrounding repository context if the model is expected to produce a full patch:
Assume `Session`, `UUID`, `select`, and `User` are already imported. Do not add imports.
✓Correct!
✗Not quite.
💡Solution
Question7
Code Review
Card 3: Code review
Task: Review a pull request for maintainability and security.
Prompt:
Act as a senior backend engineer and security-conscious code reviewer.
Review the following pull request diff. Use this rubric:
1. Correctness risks
2. Security risks
3. Maintainability issues
4. Missing tests
5. Unclear naming or API design
6. Suggestions that are nice-to-have rather than required
For each finding, output:
- severity: critical / major / minor / suggestion
- file or function
- issue
- why it matters
- recommended change
Do not rewrite the whole patch. Focus on review comments.
Here is the diff:
[DIFF]
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example review is demonstrated. The numbered rubric and the per-finding field list are instructions and an output schema, not worked input→output demonstrations. The diff is the artifact the task operates on, not a demonstration.
✗Not quite.
Hint 1
This axis is about whether an example review is demonstrated to imitate, or whether the prompt only gives instructions and a schema. Ask what role the numbered rubric and the per-finding field list play.
Hint 2
Few-shot needs a worked input→output demonstration. A numbered rubric and a per-finding output schema are instructions, not a worked example; the diff is the artifact the task operates on, not a demonstration.
💡Solution
Examples axis: zero-shot. No example review is demonstrated. The numbered rubric and the per-finding field list are instructions and an output schema, not worked input→output demonstrations. The diff is the artifact the task operates on, not a demonstration.
Question8
Code Review
Card 3 continued — same code-review prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Role priming: “Act as a senior backend engineer and security-conscious code reviewer” is an expertise persona.
Rubric-based: the numbered list of evaluation categories (correctness, security, maintainability, …) is a set of criteria to judge against.
Structured-output: the per-finding fields (severity / file / issue / why / change) are an explicitly named section schema.
Context grounding: the included [DIFF] is embedded background the review conditions on.
Not chain-of-thought: nothing asks the model to think step by step or show its reasoning.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: is there a persona, an evaluation checklist, an output schema, and concrete code to ground on?
Hint 2
Apply each rule. Role fires on “Act as a <role>”. A rubric is a list of evaluation criteria to judge against (the numbered categories). Structured-output is the named per-finding fields. Context grounding is the embedded [DIFF].
Hint 3
One distractor is the common false positive here: nothing asks the model to think step by step or show its reasoning — recheck the chain-of-thought rule before you include it.
💡Solution
Role priming: “Act as a senior backend engineer and security-conscious code reviewer” is an expertise persona.
Rubric-based: the numbered list of evaluation categories (correctness, security, maintainability, …) is a set of criteria to judge against.
Structured-output: the per-finding fields (severity / file / issue / why / change) are an explicitly named section schema.
Context grounding: the included [DIFF] is embedded background the review conditions on.
Not chain-of-thought: nothing asks the model to think step by step or show its reasoning.
Question9
Code Review
Card 3 reflection — code review prompt
Now write out your full diagnosis for Card 3 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The rubric is the primary lever. Code review is a coverage problem: without an explicit list of dimensions the model flags only what catches its eye. The rubric forces systematic coverage of correctness, security, maintainability, tests, naming, and nice-to-haves. Role priming sets the perspective and the structured schema makes findings actionable, but remove the rubric and the review becomes shallow and uneven.
☰Reference answer
“Act as a senior backend engineer and security-conscious code reviewer” is role priming.
The numbered list of review categories (correctness, security, maintainability, …) is a rubric.
The per-finding fields (severity, file, issue, …) define a structured output schema.
The included [DIFF] grounds the review in concrete code rather than asking for advice in the abstract.
☰Reference answer
Code review is a judgment task. A role helps set the perspective; a rubric improves coverage so the model doesn’t only flag what catches its eye; structured output makes the feedback actionable for the reviewer or for a tool that consumes the comments.
☰Reference answer
The model may over-report minor issues.
It may miss subtle security issues.
It may misunderstand diff context if surrounding code is missing.
It may suggest changes inconsistent with team conventions.
It may sound authoritative even when uncertain.
☰Reference answer
Static analysis.
Security scanners.
Tests.
Human review for architecture and business logic.
☰Reference answer
Separate findings into:
- Must fix before merge
- Should consider
- Questions for the author
If a finding depends on missing context, mark it as "needs context" instead of stating it as fact.
✓Correct!
✗Not quite.
💡Solution
Question10
Debugging
Card 4: Bug diagnosis with stack trace
Task: Diagnose a runtime bug.
Prompt:
I get this error:
```text
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
```
Here is the buggy function:
```python
deftotal_price(items):total=0foriteminitems:total+=item.discounted_price()returntotal```
For reference, from our pricing module `pricing/discounts.py`, here is the
method each item exposes (it returns `None` when no active discount applies):
```python
classItem:defdiscounted_price(self)->float|None:ifself._active_discountisNone:returnNonereturnself.base_price*(1-self._active_discount.rate)```
Explain the likely cause and propose a fix.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. Nothing demonstrates a desired input→output behaviour to imitate. The buggy function and the Item.discounted_price() method are artifacts to reason about, not worked example calls.
✗Not quite.
Hint 1
This axis is about whether there is a worked input→output demonstration to imitate, or only code to reason about. Ask what role the buggy function and the discounted_price() method play.
Hint 2
Few-shot needs a worked input→output demonstration. The buggy total_price function and the Item.discounted_price() method are artifacts to reason about, not example calls showing behaviour to imitate.
💡Solution
Examples axis: zero-shot. Nothing demonstrates a desired input→output behaviour to imitate. The buggy function and the Item.discounted_price() method are artifacts to reason about, not worked example calls.
Question11
Debugging
Card 4 continued — same stack-trace prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Context grounding fires: the Item.discounted_price() method from pricing/discounts.py is genuine background — distinct from the total_price function under diagnosis — that the model conditions on to see that None is an expected return. The buggy function is the task’s direct object; the pricing method is the surrounding background.
Not structured-output: “Explain the likely cause and propose a fix” is two prose imperatives joined by “and” — a task description, not an output-format spec.
Not chain-of-thought: “Explain” alone does not invoke step-by-step reasoning.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires. In particular: is there background the model conditions on that is distinct from the buggy function itself?
Hint 2
Apply each rule. Context grounding counts background you reason FROM; the artifact the task operates ON does not — so ask which of the two code blocks is which. And “explain the cause AND propose a fix” is two prose imperatives joined by “and”, which the structured-output rule says is a task description, not a format spec.
Hint 3
Two distractors are common false positives: a bare “explain” is not chain-of-thought, and there is no persona, rubric, or named tool. Recheck those rules before you include them.
💡Solution
Context grounding fires: the Item.discounted_price() method from pricing/discounts.py is genuine background — distinct from the total_price function under diagnosis — that the model conditions on to see that None is an expected return. The buggy function is the task’s direct object; the pricing method is the surrounding background.
Not structured-output: “Explain the likely cause and propose a fix” is two prose imperatives joined by “and” — a task description, not an output-format spec.
Not chain-of-thought: “Explain” alone does not invoke step-by-step reasoning.
Question12
Debugging
Card 4 reflection — stack-trace debugging
Now write out your full diagnosis for Card 4 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
Context grounding is the lever. The supplied discounted_price() definition is what lets the model see that None is an expected return rather than guessing — it turns a blind diagnosis into an evidence-based one. Remove that background and the model has to assume what discounted_price() does, which is exactly where it collapses uncertainty too early.
☰Reference answer
No worked examples are provided (zero-shot on the examples axis).
The stack trace plus the buggy total_price function are the task’s direct object.
The Item.discounted_price() method from pricing/discounts.py is embedded background context that the model conditions on — context grounding.
The task asks for both an explanation and a fix.
☰Reference answer
The error message is unambiguous and the supplied code is small, so with the discounted_price() definition in hand the model can confirm — not just guess — that None is an expected return. For a self-contained debugging task this is a reasonable prompt, though it still leaves the domain decision (should None be allowed?) open.
☰Reference answer
The model may assume too much about discounted_price() (e.g. that None always means “no discount”).
It may suggest hiding the bug by treating None as zero instead of asking whether that’s correct.
It may fail to ask whether None is valid domain behavior.
It may propose a fix without tests.
☰Reference answer
A reproduction test.
A unit test for items with missing discounted prices.
The domain decision: should None be allowed in the first place?
☰Reference answer
Diagnose this bug, but separate evidence from assumptions.
For the fix, provide two alternatives:
1. A defensive fix if `None` is expected domain behavior.
2. A stricter fix if `None` indicates invalid data.
Also suggest one regression test.
Error:
[STACK TRACE]
Relevant code:
[CODE]
✓Correct!
✗Not quite.
💡Solution
Question13
Test Generation
Card 5: Test generation (minimal)
Task: Generate tests for an existing function.
Prompt:
Write tests for this function:
```python
defis_valid_age(age:int)->bool:returnage>=18andage<=120```
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example test is demonstrated. The function shown is the unit under test — the artifact the task operates on — not a demonstration of how to write tests.
✗Not quite.
Hint 1
This axis is about whether an example test is demonstrated to imitate, or whether the prompt shows only the function to be tested. Ask what role the function source plays.
Hint 2
Few-shot needs a worked input→output demonstration. The function shown is the unit under test — the artifact the task operates on — not a demonstration of how to write tests.
💡Solution
Examples axis: zero-shot. No example test is demonstrated. The function shown is the unit under test — the artifact the task operates on — not a demonstration of how to write tests.
Question14
Test Generation
Card 5 continued — same minimal test-generation prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
None of these. There is no role, no rubric, no output-format spec, no reasoning instruction, no tool. The function source is the unit under test, not embedded background, so it does not trigger context grounding. This is a bare zero-shot instruction — and the bareness is the problem (see the reflection’s Primary lever).
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: what, if anything, is actually layered on top of the bare “Write tests for this function” instruction?
Hint 2
Apply each rule and watch the one tempting false positive: showing the function source is not context grounding, because the function is the unit under test, not background the model conditions on while producing something else. There is also no role, rubric, output spec, reasoning instruction, or named tool.
💡Solution
None of these. There is no role, no rubric, no output-format spec, no reasoning instruction, no tool. The function source is the unit under test, not embedded background, so it does not trigger context grounding. This is a bare zero-shot instruction — and the bareness is the problem (see the reflection’s Primary lever).
Now write out your full diagnosis for Card 5 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
There is no additive technique, so the only lever is the bare instruction “Write tests for this function” — and it is far too thin to carry the task. It pins down nothing about framework, coverage, or output. The real fix is not to nominate a present technique but to add one: a coverage specification (which is a rubric-plus-structured-output upgrade). The lever this task needs is missing.
☰Reference answer
No role.
No examples.
No test framework specified.
No coverage criteria specified.
No output format specified.
Showing the function source is not context grounding — the function is the unit under test, not background. The model has no information about how the project tests code.
☰Reference answer
The task is underspecified. The model doesn’t know whether to use pytest, unittest, property-based testing, table-driven tests, or project-specific conventions. It also doesn’t know what coverage is expected (boundaries? negative ages? non-int input?). Zero-shot is the wrong tool for this task; what the task needs is a coverage specification.
☰Reference answer
Tests may use the wrong framework for the project.
Boundary cases may be incomplete.
Negative ages may be missed.
The model may only test happy paths.
The model may duplicate implementation logic (assert is_valid_age(35) == (35 >= 18 and 35 <= 120)) rather than test behavior.
☰Reference answer
Running the tests.
Coverage measurement.
Mutation testing if desired.
☰Reference answer
Write `pytest` tests for `is_valid_age`.
Use table-driven tests and cover:
- below minimum: -1, 0, 17
- lower boundary: 18
- typical valid value: 35
- upper boundary: 120
- above maximum: 121
Only output the test code.
✓Correct!
✗Not quite.
💡Solution
Question16
Requirements
Card 6: Requirements clarification
Task: Turn vague stakeholder notes into acceptance criteria.
Prompt:
Act as a product-minded requirements engineer.
Turn these stakeholder notes into acceptance criteria.
Use this format:
- User story
- Acceptance criteria in Given/When/Then format
- Open questions
- Assumptions you made
Important: Do not silently invent requirements. If something is unclear, put it under Open questions.
Stakeholder notes:
"Users should be able to export their data. It should be fast and secure. Admins need some kind of overview. Also we need this for compliance next quarter."
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example acceptance-criteria document is demonstrated. The bulleted output format is a schema, not a worked example; the stakeholder notes are the input to transform.
✗Not quite.
Hint 1
This axis is about whether a filled-in example acceptance-criteria document is demonstrated to imitate, or whether the prompt gives only a format and the notes to transform. Ask what role the bulleted output format plays.
Hint 2
Few-shot needs a worked input→output demonstration. The bulleted output sections are a schema, not a worked example; the stakeholder notes are the input to transform, not a demonstration.
💡Solution
Examples axis: zero-shot. No example acceptance-criteria document is demonstrated. The bulleted output format is a schema, not a worked example; the stakeholder notes are the input to transform.
Question17
Requirements
Card 6 continued — same requirements prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Role priming: “Act as a product-minded requirements engineer” is an expertise persona.
Structured-output: the output sections (User story / Given-When-Then criteria / Open questions / Assumptions) are an explicitly named section schema.
Not rubric-based: the four sections organise the output; they are not evaluation criteria to judge the notes against.
Not context grounding: the stakeholder notes are the input to transform, not background the model conditions on while producing something else.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: look for an expertise persona and a named output section schema.
Hint 2
Apply each rule. Role fires on “Act as a <role>”. Structured-output fires when the parts are explicitly named/enumerated as a labeled list (the four output sections).
Hint 3
Two distractors are tempting: the four output sections organise the output, they are not evaluation criteria to judge the notes against (so not a rubric); and the stakeholder notes are the input to transform, not background the model conditions on (so not context grounding). Recheck both rules before including them.
💡Solution
Role priming: “Act as a product-minded requirements engineer” is an expertise persona.
Structured-output: the output sections (User story / Given-When-Then criteria / Open questions / Assumptions) are an explicitly named section schema.
Not rubric-based: the four sections organise the output; they are not evaluation criteria to judge the notes against.
Not context grounding: the stakeholder notes are the input to transform, not background the model conditions on while producing something else.
Question18
Requirements
Card 6 reflection — requirements clarification
Now write out your full diagnosis for Card 6 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The structured output — specifically the “Open questions” and “Assumptions” sections plus the “do not silently invent requirements” instruction — is the lever. It is what forces the model to separate what was given from what it inferred, which is the dominant risk for ambiguity-heavy requirements work. Role priming helps set tone, but remove the uncertainty-surfacing structure and the model will quietly invent precision.
☰Reference answer
“Act as a product-minded requirements engineer.”
The output structure is fully defined (User story / GWT criteria / Open questions / Assumptions).
“Do not silently invent requirements” directly targets hallucination.
“Open questions” and “Assumptions” force the model to separate what was given from what it inferred.
☰Reference answer
Requirements work is ambiguity-heavy. The prompt is designed to prevent the model from converting uncertainty into invented precision — which is the dominant failure mode for this task type.
☰Reference answer
The model may still over-specify compliance requirements.
It may invent export formats or performance thresholds.
It may miss security/privacy implications.
It may produce criteria that are not testable.
☰Reference answer
Stakeholder validation.
Legal/compliance review.
Product prioritization.
☰Reference answer
Mark each acceptance criterion as either directly supported by the notes or inferred.
✓Correct!
✗Not quite.
💡Solution
Question19
External API
Card 7: External API usage (Stack Exchange)
Task: Write code using an unfamiliar third-party API.
Familiar setup, deliberate framing. This prompt is the unaided zero-shot version of the Stack Exchange API task you saw in Lecture 2’s Stack Overflow Challenge and that the lecture worked through in §12.6.
Prompt:
Write Python code that uses the Stack Exchange API to get the number of questions posted per month for the last six months.
Use the correct endpoint and return a dictionary mapping `YYYY-MM` to question count.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example call, no documentation excerpt, no demonstration of the endpoint usage. The prompt assumes the model already knows the current Stack Exchange API surface.
✗Not quite.
Hint 1
This axis is about whether anyone showed the model how the API is called, or whether it is expected to already know. Scan the prompt for an example call, doc excerpt, or any demonstration.
Hint 2
Few-shot needs a worked input→output demonstration. There is no example call and no documentation excerpt here; the prompt assumes the model already knows the current API surface.
💡Solution
Examples axis: zero-shot. No example call, no documentation excerpt, no demonstration of the endpoint usage. The prompt assumes the model already knows the current Stack Exchange API surface.
Question20
External API
Card 7 continued — same Stack Exchange API prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Structured-output fires: “return a dictionary mapping YYYY-MM to question count” is a concrete output-shape spec (a named mapping with a specified key format).
Not tool-augmented: the prompt names no tool or documentation to invoke; it assumes the API surface is already known.
Not context grounding: no documentation or background is embedded.
The structured-output box is correctly checked — but note (in the reflection) that the shape of the dict is not where this prompt fails. The central risk is whether the data inside the dict is real.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: is the output shape specified, and is any tool or documentation named?
Hint 2
Apply each rule. Structured-output fires on a concrete output-shape spec — “return a dictionary mapping YYYY-MM to question count” is a named mapping with a specified key format. Tool-augmented must NAME a tool or resource to invoke; merely needing the API does not count, and no docs are embedded for grounding.
💡Solution
Structured-output fires: “return a dictionary mapping YYYY-MM to question count” is a concrete output-shape spec (a named mapping with a specified key format).
Not tool-augmented: the prompt names no tool or documentation to invoke; it assumes the API surface is already known.
Not context grounding: no documentation or background is embedded.
The structured-output box is correctly checked — but note (in the reflection) that the shape of the dict is not where this prompt fails. The central risk is whether the data inside the dict is real.
Question21
External API
Card 7 reflection — Stack Exchange API
Now write out your full diagnosis for Card 7 in plain prose. You met this prompt before in Lecture 2’s Stack Overflow Challenge and the lecture walked through it in §12.6; this is the formal strategy + failure-mode + improvement version. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The only present technique is the structured-output dict spec, and it is not the lever that matters — it shapes the container, not the contents. The quality of this output hinges on a technique that is absent: grounding the model in the real API (documentation or a tool call). So the honest answer is that the present lever (the dict shape) is weak, and the task’s true lever — context grounding / tool use — is missing. That gap is exactly what makes this an anti-pattern.
☰Reference answer
No documentation is provided.
No tool use is requested.
The prompt assumes the model already knows the current Stack Exchange API surface.
The “return a dictionary mapping YYYY-MM to question count” line is a structured-output spec, but it shapes the container, not the contents — the central failure mode is whether the data inside the dict is real.
☰Reference answer
This task depends on an external API surface that changes over time. The model may hallucinate an endpoint, parameter, pagination behavior, rate limit, or response schema — confidently, in plausible-looking Python. Zero-shot is the wrong tool for tasks whose central risk is hallucinating facts about a changing interface.
☰Reference answer
Factual hallucination: a non-existent endpoint.
Structural hallucination: plausible but wrong response fields.
Missing pagination or date handling.
Code that looks reasonable but fails at runtime.
☰Reference answer
Reading official API documentation.
Running the code against the API.
Handling HTTP errors and rate limits.
Tests with mocked API responses.
☰Reference answer
Use the official Stack Exchange API documentation provided below. Do not invent endpoints or fields.
First identify which documented endpoint can answer the task. If the API does not provide monthly aggregation directly, say so and compute monthly counts from available data.
Return:
1. A short explanation of the endpoint used.
2. Python code.
3. Assumptions and limitations.
4. Suggested mock tests.
Documentation excerpt:
[PASTE RELEVANT DOCS]
✓Correct!
✗Not quite.
💡Solution
Question22
Structured Extraction
Card 8: Structured output for pipeline integration
Task: Extract machine-readable metadata from a bug report.
Prompt:
Extract the following fields from the bug report and return JSON:
- title: string
- severity: one of [low, medium, high, critical]
- affected_component: string or null
- reproduction_steps: array of strings
- expected_behavior: string or null
- actual_behavior: string or null
- missing_information: array of strings
Bug report:
"Checkout sometimes crashes after I apply a coupon. I expected the discount to be applied, but instead I got a 500 error. This happened yesterday on mobile. I don't know if it also happens on desktop."
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example extraction is demonstrated. The field list with types and the enum are an output schema, not a worked input→output example. The bug report is the input to transform.
✗Not quite.
Hint 1
This axis is about whether a filled-in example record is demonstrated to imitate, or whether the prompt gives only a schema and the report. Ask what role the field list and enum play.
Hint 2
Few-shot needs a worked input→output demonstration. The field list with types and the enum are an output schema, not a worked example; the bug report is the input to transform, not a demonstration.
💡Solution
Examples axis: zero-shot. No example extraction is demonstrated. The field list with types and the enum are an output schema, not a worked input→output example. The bug report is the input to transform.
Question23
Structured Extraction
Card 8 continued — same bug-report extraction prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Structured-output fires strongly: an explicit JSON request, named fields with types, and an enum of allowed severity values — a named serialization with an enumerated schema.
Not rubric-based: the fields are a data schema to fill, not evaluation criteria to judge against.
Not context grounding: the bug report is the input to transform, not background.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: what names the serialization and enumerates the allowed values?
Hint 2
Apply each rule. Structured-output fires on a named serialization (an explicit JSON request) with named, typed fields and an enum of allowed values. Watch the false positive: a data schema to fill is not a rubric of evaluation criteria, and the bug report is the input to transform, not background (so not context grounding).
💡Solution
Structured-output fires strongly: an explicit JSON request, named fields with types, and an enum of allowed severity values — a named serialization with an enumerated schema.
Not rubric-based: the fields are a data schema to fill, not evaluation criteria to judge against.
Not context grounding: the bug report is the input to transform, not background.
Question24
Structured Extraction
Card 8 reflection — bug-report extraction
Now write out your full diagnosis for Card 8 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The structured-output schema is the lever: the named JSON fields, types, and the severity enum are what make the output pipeline-consumable. Remove the schema and you get prose that no downstream system can ingest. Note that the schema biases the model toward valid JSON but does not enforce it — that is the enforcement layer’s job (see Outside the prompt).
☰Reference answer
Explicit JSON request.
Field names and types are specified.
Enum values are specified for severity.
☰Reference answer
The output is intended to be machine-readable. A structured format helps integrate the model output into a downstream workflow — but prompting for JSON is not the same as enforcing JSON. The prompt biases the model toward the schema; it does not guarantee compliance.
☰Reference answer
Invalid JSON.
Missing required fields.
severity may be guessed without enough evidence.
The model may put inferred information where null is more appropriate.
The model may violate enum constraints.
☰Reference answer
JSON schema validation.
Retry/repair loop if invalid.
Human review for severity.
Possibly constrained decoding or a structured-output API (§15.1).
☰Reference answer
Return only valid JSON matching this schema. Use `null` when the report does not provide the information. Do not infer severity beyond the evidence; if uncertain, use `medium` and include the uncertainty in `missing_information`.
[JSON SCHEMA]
✓Correct!
✗Not quite.
💡Solution
Question25
Architecture
Card 9: Architecture decision support
Task: Compare architectural options.
Prompt:
Act as a pragmatic software architect.
We need to decide how to process image uploads in our web application.
Compare these options:
1. Process synchronously in the request.
2. Store the image and process it in a background worker.
3. Send it to an external image-processing service.
Use this decision rubric:
- latency
- implementation complexity
- operational complexity
- failure handling
- scalability
- cost
- security/privacy
- developer experience
Output:
1. A comparison table.
2. A recommendation for a small startup team.
3. Conditions under which your recommendation would change.
4. Assumptions you are making.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example comparison is demonstrated. The listed options, the rubric, and the output sections are instructions and schema, not worked examples to imitate.
✗Not quite.
Hint 1
This axis is about whether a filled-in example decision is demonstrated to imitate, or whether the prompt gives only options, a rubric, and an output schema. Ask what role those three elements play.
Hint 2
Few-shot needs a worked input→output demonstration. The listed options, the rubric, and the output sections are instructions and schema, not a worked comparison to imitate.
💡Solution
Examples axis: zero-shot. No example comparison is demonstrated. The listed options, the rubric, and the output sections are instructions and schema, not worked examples to imitate.
Question26
Architecture
Card 9 continued — same architecture-decision prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Role priming: “Act as a pragmatic software architect” is an expertise persona.
Rubric-based: the eight-criterion list (latency, complexity, cost, security, …) is a set of evaluation dimensions to judge each option against.
Structured-output: the four numbered output sections (table, recommendation, conditions, assumptions) are an explicitly named section schema.
Not context grounding: the three options are part of the task, not embedded background about the model’s own application.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: find the persona, the evaluation dimensions, and the output schema.
Hint 2
Apply each rule. Role fires on “Act as a <role>”. A rubric is the list of evaluation criteria to judge each option against (the eight criteria). Structured-output is the named, numbered output sections.
Hint 3
One distractor is the common false positive: the three options are part of the task statement, not embedded background about the model’s own application — recheck the context-grounding rule before you include it.
💡Solution
Role priming: “Act as a pragmatic software architect” is an expertise persona.
Rubric-based: the eight-criterion list (latency, complexity, cost, security, …) is a set of evaluation dimensions to judge each option against.
Structured-output: the four numbered output sections (table, recommendation, conditions, assumptions) are an explicitly named section schema.
Not context grounding: the three options are part of the task, not embedded background about the model’s own application.
Question27
Architecture
Card 9 reflection — architecture decision support
Now write out your full diagnosis for Card 9 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The rubric is the primary lever. Architecture decisions are trade-off decisions, and the eight-criterion rubric is what prevents a one-dimensional answer (“just use a worker”). Role priming sets tone and the output schema makes the comparison legible, but remove the rubric and the model evaluates the options on whatever axis it happens to favour.
☰Reference answer
“Act as a pragmatic software architect.”
Options are explicitly listed (not generated by the model).
An eight-criterion rubric is specified.
The output sections (table, recommendation, conditions, assumptions) are specified.
☰Reference answer
Architecture decisions are trade-off decisions. A rubric and a structured comparison help prevent a one-dimensional answer (“just use a worker”). Asking for explicit assumptions makes the model’s hidden context visible, which is essential for a recommendation a human will act on.
☰Reference answer
Recommendation may depend heavily on unstated context.
The model may understate operational complexity.
Security/privacy risks may be generic.
Cost assumptions may be wrong.
☰Reference answer
Real traffic estimates.
Cost modeling.
Security review.
Team capability assessment.
An architecture decision record reviewed by humans.
☰Reference answer
Before recommending, list the three most important missing pieces of context that could change the decision.
✓Correct!
✗Not quite.
💡Solution
Question28
Security
Card 10: Security-sensitive code (JWT middleware)
Task: Add authentication middleware.
Prompt:
Write Express.js middleware that checks whether a request has a valid JWT.
If the token is valid, attach the user ID to `req.userId` and call `next()`.
If invalid, return HTTP 401.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example middleware is demonstrated. The prompt is a bare behavioural instruction.
✗Not quite.
Hint 1
This axis is about whether an example middleware is demonstrated to imitate, or whether the prompt gives only a behavioural instruction. Scan for any worked demonstration.
Hint 2
Few-shot needs a worked input→output demonstration. There is none here — the prompt is a bare behavioural instruction describing what the middleware should do.
💡Solution
Examples axis: zero-shot. No example middleware is demonstrated. The prompt is a bare behavioural instruction.
Question29
Security
Card 10 continued — same JWT middleware prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
None of these. No role, no rubric, no output-format spec, no reasoning instruction, no embedded background, no tool. “Attach the user ID to req.userId” describes behaviour, not an output schema. This is a bare zero-shot prompt — for a security-critical task, which is exactly the problem (see the reflection).
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: is anything actually layered on top of the bare behavioural instruction?
Hint 2
Apply each rule and watch the false positive: “attach the user ID to req.userId” describes behaviour, it is not an output-format spec. There is no role, rubric, reasoning instruction, embedded background, or named tool either.
💡Solution
None of these. No role, no rubric, no output-format spec, no reasoning instruction, no embedded background, no tool. “Attach the user ID to req.userId” describes behaviour, not an output schema. This is a bare zero-shot prompt — for a security-critical task, which is exactly the problem (see the reflection).
Question30
Security
Card 10 reflection — JWT middleware
Now write out your full diagnosis for Card 10 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
There is no additive technique, so the only lever is the bare instruction — and for a security-critical task it is dangerously thin. It specifies the happy path and nothing about the security policy (algorithms, issuer/audience, expiry, error behaviour). What the task actually needs is layered technique (a security-requirements rubric, plus enforcement and review) — the present lever is not enough.
☰Reference answer
No role.
No examples.
No library specified.
No algorithm/issuer/audience constraints.
No error-handling, logging, or expiration policy.
☰Reference answer
Authentication is security-critical. A generic zero-shot prompt may omit algorithm restrictions, issuer/audience checks, expiration handling, key rotation, error behavior, logging rules, and dependency choices. Whatever the model produces will look like JWT middleware — but the gap between “looks correct” and “is correct” is exactly where security bugs live. Zero-shot is the wrong tool when the task’s failure modes are subtle and high-stakes.
☰Reference answer
Accepting unsigned tokens or the wrong algorithms.
Not checking expiration.
Not checking issuer/audience.
Leaking token details in errors.
Attaching unvalidated claims to the request.
Using outdated or unsafe libraries.
☰Reference answer
Security review.
Dependency review.
Tests with valid, expired, malformed, wrong-audience tokens.
Static analysis and secret-handling checks.
Runtime configuration management.
☰Reference answer
Act as a security-conscious Node.js backend engineer.
Write Express.js middleware for JWT authentication using `[SPECIFIC LIBRARY]`.
Security requirements:
- Verify signature.
- Accept only algorithm `[ALGORITHM]`.
- Validate issuer `[ISSUER]`.
- Validate audience `[AUDIENCE]`.
- Reject expired tokens.
- Do not log token contents.
- Return a generic 401 on failure.
- Attach only the validated subject claim to `req.userId`.
Also provide test cases for:
- valid token
- expired token
- wrong issuer
- wrong audience
- malformed token
- missing token
If any required configuration is missing, state it explicitly instead of inventing it.
✓Correct!
✗Not quite.
💡Solution
Question31
Agentic
Card 11: Agentic repository task
Task: Ask an AI coding agent to modify a repository.
Prompt:
Implement issue #214.
Make the necessary changes and commit them.
Stem note. A prompt that merely implies tool use (“commit them”) does not name a tool, so per the tool-augmented decision rule the tool box stays unchecked here. The real point — that this task NEEDS scoped tool use, branch protection, and review — belongs in the reflection, not the graded checkboxes.
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No demonstration of any kind. Two terse imperatives, nothing to imitate.
✗Not quite.
Hint 1
This axis is about whether there is anything to imitate, or just two bare imperatives. Scan for a worked demonstration.
Hint 2
Few-shot needs a worked input→output demonstration. There is nothing of the kind here — only two terse imperatives, with nothing shown to imitate.
💡Solution
Examples axis: zero-shot. No demonstration of any kind. Two terse imperatives, nothing to imitate.
Question32
Agentic
Card 11 continued — same agentic prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
None of these. No role, rubric, output schema, reasoning instruction, or embedded context. Crucially, not tool-augmented: “commit them” implies tool use but names no tool and scopes nothing — per the tool-augmented decision rule, an implied action is not a named tool. The danger of this prompt lives in the reflection (it needs scoped tools, branch protection, review), not in any checked box.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires. The tempting one: does the prompt NAME a tool to invoke, or merely imply that work will happen?
Hint 2
Apply the tool-augmented rule carefully: it must NAME a tool; merely implying tool use (“commit them”) does not count. There is also no role, rubric, output schema, reasoning instruction, or embedded background — so weigh the “None of these” option against what is literally printed.
💡Solution
None of these. No role, rubric, output schema, reasoning instruction, or embedded context. Crucially, not tool-augmented: “commit them” implies tool use but names no tool and scopes nothing — per the tool-augmented decision rule, an implied action is not a named tool. The danger of this prompt lives in the reflection (it needs scoped tools, branch protection, review), not in any checked box.
Question33
Agentic
Card 11 reflection — agentic repository task
Now write out your full diagnosis for Card 11 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
There is no additive technique, so the only lever is the bare instruction — and it is far too thin for an autonomous agent with write access. The task’s true levers are absent and external: scoped tool use, task boundaries, a stop-before-commit gate, branch protection, and review. The graded checkboxes correctly show “None,” and that emptiness is precisely the danger.
☰Reference answer
Very little context.
No acceptance criteria.
No constraints.
No testing requirements.
No permission boundaries.
Allows commit without review.
Note that this prompt implies tool use (the agent must read the issue, edit files, commit) — but it does not name or scope any tool. Implied tool use is not tool-augmented prompting; tool-augmented prompting names tools and scopes their use.
☰Reference answer
Agentic coding tasks can modify many files. Without boundaries and validation requirements, the agent may make broad, low-quality, or incorrect changes — and then commit them. The danger compounds because the prompt is short and confident: the agent has no reason to slow down or ask. For an autonomous tool with write access to your repo, a bare prompt is the wrong tool.
☰Reference answer
Misunderstanding the issue.
Over-modifying unrelated files.
Not running tests.
Creating a commit that passes superficially but breaks behavior.
Ignoring project conventions.
Failing silently on edge cases.
☰Reference answer
Branch protection.
CI.
Human review before merge.
Test suite.
Commit hooks.
☰Reference answer
Work on issue #214 in a new branch.
Before editing:
1. Read the issue description.
2. Identify the relevant files.
3. Summarize the intended change and ask for confirmation if the scope is ambiguous.
While editing:
- Make the smallest change that satisfies the issue.
- Do not refactor unrelated code.
- Follow existing project style.
- Add or update tests that fail before the fix and pass after it.
Before finishing:
- Run the relevant test subset.
- Run the formatter and linter.
- Provide a summary of changed files, tests run, and remaining risks.
- Do not commit unless explicitly instructed after review.
✓Correct!
✗Not quite.
💡Solution
Question34
Anti-patterns
Card 12: Overuse of chain-of-thought
Task: Classify a bug report.
Prompt:
Think step by step in great detail. Show all your reasoning.
Classify this bug report as one of:
- UI
- backend
- database
- infrastructure
- unclear
Bug report:
"The save button turns gray and nothing happens after I upload a profile picture."
Does this prompt use worked examples? (the examples axis)
✓Correct!
Examples axis: zero-shot. No example classification is demonstrated. The five-way label set is the allowed-output enum, not a worked demonstration.
✗Not quite.
Hint 1
This axis is about whether an example classification is demonstrated to imitate, or whether the prompt gives only the label set. Ask what role the five-way list plays.
Hint 2
Few-shot needs a worked input→output demonstration. The five-way label set is the allowed-output enum, not a worked classification to imitate.
💡Solution
Examples axis: zero-shot. No example classification is demonstrated. The five-way label set is the allowed-output enum, not a worked demonstration.
Question35
Anti-patterns
Card 12 continued — same chain-of-thought prompt as above.
Which ADDITIVE techniques are present? Select all that apply.
Select all that apply:
✓Correct!
Chain-of-thought: “Think step by step in great detail. Show all your reasoning” is an explicit reasoning instruction.
Structured-output: the five-way enum (UI / backend / database / infrastructure / unclear) constrains the output to one of a fixed set of allowed values.
Not rubric-based: the enum is the allowed answers, not criteria for judging them.
The CoT box is correctly checked — but the reflection’s appropriateness analysis is where you decide that CoT is overkill for a five-way classification.
✗Not quite.
Hint 1
Go option by option and ask which printed trigger each one fires: what invokes reasoning, and what constrains the answer to a fixed set of values?
Hint 2
Apply each rule. Chain-of-thought fires on an explicit “think step by step” / “show your reasoning” instruction. Structured-output fires on an enum of allowed values (the five-way label set). Watch the false positive: the enum is the allowed answers, not evaluation criteria, so it is not a rubric.
💡Solution
Chain-of-thought: “Think step by step in great detail. Show all your reasoning” is an explicit reasoning instruction.
Structured-output: the five-way enum (UI / backend / database / infrastructure / unclear) constrains the output to one of a fixed set of allowed values.
Not rubric-based: the enum is the allowed answers, not criteria for judging them.
The CoT box is correctly checked — but the reflection’s appropriateness analysis is where you decide that CoT is overkill for a five-way classification.
Now write out your full diagnosis for Card 12 in plain prose. After you submit, the reference answer for each field reveals.
Write your answer in each field. After you submit, the reference answer reveals. (No automatic grading; your text stays in your browser only.)
☰Reference answer
The structured-output enum is the lever that actually helps: it forces a clean, machine-checkable label. The chain-of-thought instruction is present but counter-productive here — for a five-way classification it adds verbosity and false confidence without improving the label. So the present lever worth keeping is the enum; the CoT is the technique to drop.
☰Reference answer
“Think step by step in great detail.”
“Show all your reasoning.”
The five-way label enum (UI / backend / database / infrastructure / unclear) is a structured-output constraint.
No demonstrations of the classification (zero-shot on the examples axis).
☰Reference answer
The task is a simple classification with ambiguity baked into the report. A long visible reasoning chain is not necessary and may create false confidence — and the chain is not even an audit trail of how the model actually decided (see §13.5 and §15.3). What matters for this task is the label, the confidence, and the missing information — not detailed reasoning.
☰Reference answer
Verbose answer.
Rationalization of a weak guess.
Overconfident classification.
Hidden ambiguity gets transformed into an explanation.
☰Reference answer
Reviewing the classification against a labeled sample.
Routing to a human when confidence is low.
☰Reference answer
Classify the bug report as one of:
- UI
- backend
- database
- infrastructure
- unclear
Return:
- label
- confidence: low / medium / high
- one-sentence rationale
- what information would make the classification more certain