07 Lecture 7 Quiz: The System Around the Model

The lecture’s diagnostic sorts every behaviour into exactly one bin: a file that got loaded or didn’t is an outer-box event, governed by rules, file paths, and parsing, while distributions and attention are inner-box. A prompt changes only the tokens going into the model, so it cannot touch a wrapper rule that decides which files to splice in. The most tempting wrong answer is A, because attention dilution is real, but the lecture explicitly classes “a file that got loaded or didn’t” as the outer box, not the inner box where attention lives. C reflects the persona misconception that “the model decides,” which the lecture rejects since the model does not even know what files exist; D is simply false, because the wrapper does inject loaded file contents as tokens. The lecture names exactly this case as the most common failure mode: people write ever-more-elaborate prompts to fix orchestration problems.

The lecture pins the error on interface design: a chat window shows one name, one voice, one assistant, so every capability that appears in the conversation is attributed to that persona, and the wrapper, where most engineering decisions live, becomes invisible. The most tempting wrong answer is B, but the lecture is emphatic that the model is passive and persists nothing, so the memory and file access actually live in the wrapper, not the model. C inverts the lecture’s claim, which insists the orchestration layer is simple, ordinary code a developer reads in an afternoon, not too complex to reason about. D introduces marketing, a cause the lecture never mentions; it locates the error squarely in the single-persona UI, a choice it calls deliberate and reasonable but also the reason mental models err in the same direction.

The lecture pins the KV cache down precisely: it saves recomputation within one generation, changing what must be recomputed, not what must be sent, while the model itself is stateless and the whole sequence is re-sent in full on every call. D is the most tempting distractor because it restates the common belief that the model “remembers,” but the lecture insists nothing persists between calls and continuity lives in the sequence, not the model. B wrongly turns the cache into a persistent session object, the exact thing the lecture says does not exist, and A invents a system-prompt-only cache the lecture never describes.

The lecture states there is no instruction channel anywhere: the only structural marking the model gets is the role labels, and a label like [system] is influential not because the architecture enforces it but because post-training taught the model to weight those turns heavily; strip that learned preference and all three are just text in a sequence. The most tempting wrong answer is C, because it sounds like the labels still do something special, but the lecture is explicit that labels are rendered as special tokens the model reads, not metadata around the text. B invents a dedicated channel the lecture says does not exist, and A substitutes authorship for the actual mechanism, which is learned weighting, not who wrote the block.

The lecture states plainly that the neural network does not decide which instruction files to load — it cannot ask for more context; the orchestration layer decides by rules as dumb as directory traversal. This mechanical dumbness is a feature because you can know, exactly and without running any model, which files entered a session: list the ancestors of the working directory and the subdirectories the tools touched. A is the most tempting wrong answer because it sounds like sensible engineering, but the lecture is emphatic that the model cannot ask for context and the closest it comes is indirect (a tool call it emitted asking for a file, not for the conventions). C wrongly says the model loads files and misattributes the benefit to speed rather than auditability. D invents a memory/learning mechanism the lecture never describes — the rules are stateless if-statements over a directory tree.

Because the session never started in or below tests/ and no tool call touched that subtree, neither rule one (the start-up ancestor walk) nor rule two (on-demand injection) fired — so the file’s tokens were never in the array. The lecture is explicit that such a convention “is not weakly attended; it is absent,” and the orchestration-layer fix is to move the file up the tree or start where the walk will find it. A is the most tempting wrong answer because it names a real failure mode, but “loaded but outcompeted” requires the tokens to already be in the sequence — here they never arrived, so its fix (shorten, front-load) addresses the wrong half of the system. C invents an overwrite story that belongs to a different chapter and is not what happened. D assumes the model has agency to read and ignore, but the conventions were never delivered, so no sterner phrasing could help.

The lecture resolves the pytest anecdote entirely through the five-step loop: in the first session the model wrote a memory file recording the preference, and at the start of the second the orchestration layer injected that file’s tokens before any user input, so attention did the rest. Persistence lives on disk, not in the model; each conversation starts with a model that has never met you. The most tempting wrong answer is B: it feels like learning, but the weights were frozen when training ended and nothing you say changes them. C is wrong because the KV cache is only a compute-saving copy derivable from the current sequence, not an accumulator across sessions, and D is wrong because the model never reaches for these files at all — the orchestration layer reads them, by rule.

The lecture’s whole point about memories born wrong is persistence: the model hallucinates in writing exactly as it hallucinates in chat, but a chat hallucination scrolls away while a hallucinated memory is re-injected into every future conversation until someone deletes it. The most tempting wrong answer is B, but the lecture stresses the opposite — memory files are written with the same sampling machinery as everything else, so the file-write path is no more error-prone, only more durable. C invents a special authority the lecture never grants; a memory only wears the authority of an instruction by virtue of its early position, it does not become uncorrectable. A is the inverse of the stated reality: nothing verifies a memory against the codebase, which is exactly why the danger exists.

The lecture’s central point is that a token sequence only grows: editing the file on disk does not touch the sequence, and the only thing re-reading can do with the fresh contents is append them, so v2 lands far down (~position 10,500) as a tool result while v1 remains at its original early position (~500). Both copies then draw attention at every step, and the per-token sampled outcome produces intermittent compliance. B is the most tempting wrong answer because it matches a programmer’s state-machine intuition that a save is a destructive overwrite — but the lecture stresses “there is no overwrite in a token sequence, only accumulation,” and nothing in the context was replaced. A wrongly attributes agency and judgment to the model, when the lecture frames the behaviour as softmax arithmetic over two genuinely-present rules, not defiance. D invents a replacement-plus-cache mechanism the lecture never describes; the old rule persists because its tokens were never removed, not because anything was cached back in.

The lecture gives exactly two concrete costs of mid-array surgery: it invalidates the cached KV prefix for every position after the edit, and it silently rewrites a history the model’s own intermediate outputs were conditioned on — ten turns of v1-compliant work. D names both, so layers append and only append. B is the most tempting distractor because it sounds like a hard mechanical limit, but the lecture explicitly says the array “is its data structure, not the model’s,” so the layer does have write access — lack of access is not the obstacle, the two costs are. A invents a position-numbering decode failure the lecture never mentions; the obstacle is cache invalidation and falsified history, not an index gap. C is wrong because a restart builds a fresh sequence rather than deleting tokens from the existing one, and the lecture frames restart as the clean resolution, not as the reason deletion is avoided.

The advantage is purely positional: a skill is spliced in at the moment it is invoked, so it arrives near the end of the sequence beside the current query, exactly the recency-advantaged region, whereas CLAUDE.md loads at the top and drifts toward the lost-in-the-middle zone as the conversation grows. B is the most tempting wrong answer, but the lecture is emphatic that there is no special channel, a skill’s body is plain [user]-side tokens the model cannot distinguish from a typed prompt, so no trained priority is involved. A invents a token-size mechanism the lecture never states; position, not size, is what matters. D wrongly gives the model agency, but the orchestration layer, not the model, decides when and where to inject, and the model never reaches for files.

The lecture is explicit that the model’s entire contribution is tokens: a tool call is produced “exactly like every other token,” with a full forward pass, logits, softmax, and the sampling pipeline, and the orchestration layer is what intercepts those tokens and executes the actual read or run. The most tempting wrong answer is B, because an agent’s behaviour feels like direct action, but the lecture insists there is “no separate action circuit and no hidden API” and “the model has no hands.” C fails for the same reason and adds the false idea of a built-in API. D describes the old well-fed-autocomplete world the chapter opens by rejecting, where the human carries out the changes by hand.

The lecture’s load-bearing sentence is that what an AI system can actually do is determined by its tool surface, not its intelligence, so the defense is never a sterner CLAUDE.md but a narrower tool surface, scoped permissions, schema validation on every call, and a human gate in front of anything irreversible — enforcement at the interface, not influence in the prompt. B is the single most tempting wrong option because students reach for a written rule, but the lecture explicitly rules it out: for adversarial inputs the influence layer is assumed defeated, and the model that emits the call is completing the most plausible continuation, not breaking a rule it could be told to follow. C fails because more intelligence does not shrink the tool surface and the model is not malfunctioning, and D fails because the harm is the tool surface that lets a plausible continuation execute, regardless of which tokens triggered it.

The lecture distinguishes two things: fitting in the window is binary (you are in or out), but being attended to is shaped by the learned scores — per head and per query position the softmax weights sum to one, so adding low-value tokens can pull weight away from the ones that matter, though a token the model scores highly keeps most of its attention regardless of context length. The most tempting wrong answer is B: recency does correlate with surviving eviction (keep-recent-N), but the lecture frames the currency as attention, not position, and never claims the model weights recent tokens more by design. C names a real fact (the vendor takes its share first) but that is about who controls the budget, not what the real currency is, and A invents a length-based prioritisation the lecture never states.

The lecture is explicit that nothing in the arithmetic imposes a ceiling — attention multiplies a sequence of any length — so three external constraints force the declaration: the position table or formula was only trained up to some length, the quadratic N×N attention cost holds that training length down, and the KV cache grows per token and must fit in accelerator memory. B is the tempting trap because it sounds mechanistic, but the lecture says the opposite: every matrix multiplication stays perfectly defined on longer sequences. A invents failures the lecture never mentions, and D contradicts the lecture’s framing that the number genuinely reflects “trained, affordable, and supported up to here,” not a commercial cap dressed up afterward.

The lecture states plainly that when a session feels like the model “forgot,” what usually happened is the orchestration layer evicted the information — via sliding window, keep-recent-N, or lossy summarisation — and that eviction is a system decision, not a memory failure inside the weights. C is the most tempting wrong answer because it sounds like the same outcome, but the lecture is emphatic that the in-session model is never consulted: it is stateless and simply processes whatever sequence arrives, so deciding what arrives is the active layer’s job, never the model’s. A misdescribes the mechanism entirely — nothing is written to or overwritten in the frozen weights during a conversation — and D invents a cache-corruption failure the lecture never claims.

The lecture states the enforcement layer is two things at once: a gate that can refuse to merge (deterministic on the way out) and a feedback signal whose nonzero exit code and violation list come back as tokens that condition the next forward pass like any other tokens (influence on the way back in). B is the most tempting wrong answer, but the lecture never says the model overrides the verdict; the check’s determinism as a gate is unchanged, only its role shifts because tool results are tokens. C inverts the order the lecture gives, and the category does not depend on the prompt requesting it. D invents a rewriting step the lecture never mentions; the same violation tokens re-enter unchanged.

The lecture states determinism cannot be prompted into the model and has to be bolted on from outside, because generation ends in a sample from a distribution and no instruction-file rewrite changes that final step from probable to guaranteed; only the enforcement layer truncates the distribution. B is the trap for a student who thinks better prompting eventually crosses into certainty, but the lecture is explicit that the influence layer is probabilistic by nature, not by implementation flaw. A describes a real influence-layer technique (ordering plus lost-in-the-middle) that still only narrows, never truncates. D contradicts the lecture’s claim that instruction tokens and question tokens are the same kind of thing with no special command channel.

The lecture decomposes the divergence into four bookkeeping differences (which files are read, [system] vs [user] placement, loading point, and surrounding context), but they all collapse to one mechanism: the same model under different orchestration is mechanically processing different token sequences, and since Lecture 5 you have known the token sequence is the only input the model has — “same weights, different wrapper, different token sequence, different behaviour.” The most tempting wrong answer is A: the lecture stresses it is the same weights and even the same model family, with the wrapper, not the weights, accounting for the difference. C imports the illusion of memory the lecture explicitly debunks — a tool that does not read your file behaves as if the instructions were never written, with no cross-session recall. D invents a paraphrasing step the lecture never describes; placement and loading change where tokens land, not their wording.

The lecture frames it precisely: Act 2 asked “why does the system behave this way?” — explanation, which is finished the moment the mechanism is named — while Act 3 asks “how do we engineer reliable software work on top of that behaviour?”, and engineering is never finished that way, because it wants budgets turned into policy, “usually” backed by “always,” and observations upgraded to measurements. The most tempting wrong answer is B, but the lecture is emphatic that nothing about the system changes between this week and next: tokenisation still hides characters, attention still dilutes, the sequence still only grows — only the question changes. C is wrong because the board comes down with eleven mechanisms and zero appeals to magic; no behaviour mystery is left for Act 3 to explain. A overshoots: Act 3 does not discard mechanism — it tells you to bring today’s vocabulary along, because the system around the model does not stop mattering once someone starts measuring.