Case study · Adapting a plan you already committed to

Eight Simulations: When the Evidence Changes

Hindsight-perfect design is easy to grade and useless to practise. These eight simulations give you a committed plan, then move the ground under it — and score what you do next, not what you would have designed if you had known.

Each simulation: 25–40 minAll eight: two sittingsPrerequisite: the capstone packet and lessons 0046, 0047, 0049, 0050

Mission tie-in: a lead's real work is not the first plan. It is the third one — made in public, with a named trade, while the original commitments are still on record. These simulations rehearse that specific act.

How a simulation works

Each card follows the same four beats.

Read the starting position only. Every simulation begins from the same worked initiative — the returns-resolution pilot — and names which artifacts are in force. Those are real documents with real numbers; open them before you read further.
Read the injection. An email, a message, a research summary, a metrics table: the shape evidence actually arrives in. It is incomplete on purpose. Some of what you need is missing, and noticing what is missing is part of the exercise.
Commit before you reveal. Write your five responses down, in a file, with a timestamp. Then — and only then — open the reveal. This is not ceremony. If you read two worked responses first, you will find yourself agreeing with one of them and you will learn nothing about your own judgment, because you will never see the answer you would actually have given.
Compare, and score the difference. The reveal shows two defensible responses, deliberately different from each other. Your answer will match neither. Score it against the criteria below, not against the reveal.

The five things every response must contain

What changed. Name the specific assumptions, drivers, constraints and artifacts the new evidence has moved — by identifier, not by theme. "This affects the architecture" is not an answer; "this falsifies assumption A3 and makes driver AV-1's measure untestable" is.
Continue, change, pause, or stop. One of the four, said out loud, with the reason. Continue means the evidence does not move the decision and you can say why. Change means the opportunity stands but the mechanism does not. Pause means the next decision needs evidence you can buy quickly. Stop means the opportunity is too small, the guardrail cannot be protected, or the strategy moved.
Scope and forecast, updated explicitly. What comes out, what stays, and the new forecast as a range with a confidence and a named moving variable. A response that changes the design without changing the forecast has hidden the cost somewhere.
Supersede, do not rewrite. An affected ADR gets a successor that references it and a status of superseded by; the frame gets a version bump with a changelog line; a driver's measure changes with the date and the reason attached. History stays readable. Editing yesterday's decision so it looks like today's is how an organization loses the ability to learn.
Communicate at two depths. Three sentences for a non-technical audience — outcome, change, ask. Five bullets for engineers — mechanism, gate, risk, ownership, what not to do. Neither may contain a claim the other contradicts.

Artifacts in force at the start of every simulation

Artifact	What it currently says
Initiative frame v0.4	Median resolution 72 h → under 4 h for eligible domestic returns; support contacts 41% → −25%; guardrails: fraud loss no worse than baseline, manual review queue ≤ 50/day, WCAG 2.2 AA on critical flows. Pilot by 30 September. Non-goals: exchanges, international, guest checkout.
Assumption map	A1 customers will self-serve; A2 policy rules are codifiable; A3 the carrier can meet the label latency target; A4 status visibility is secondary to decision speed; A5 pilot volume is around 40 rps at peak.
Driver set	AV-1 availability (carrier unavailable 30 min → accept requests, 99% of labels within 15 min of recovery); PF-1 performance (40 rps, eligibility decision p95 < 800 ms); SE-1 security (no cross-customer disclosure); MO-1 modifiability (one new policy rule, one engineer, one day); OP-1 operability (label failures > 2% for 5 min → identify and disable within 10 min).
ADR-0003	Persist label intent before calling the carrier; process from a worker; idempotent carrier call; expose a pending state. Status: accepted, 12 September, owner returns team.
Story map	Slice 1 request and decision · slice 2 refund · slice 3 label · slice 4 status visibility · slice 5 agent override.
Rollout contract	team → staff → 1% → 10% → 50% → 100%, each stage with entry, observation window, advance threshold and reversal.
Delivery map and forecast	70% confident of reaching the eligible domestic cohort between 23 and 30 September. Fixed: date, outcome, guardrails, team size. Moving variable: scope.
Later ADRs	The capstone log shows ADR 0001–0005 (0005 is the manual warehouse-receipt confirmation). The simulations additionally assume two decisions recorded during build and not shown in the packet: ADR-0006, reconcile refunds against the payments export nightly; and ADR-0007, the single-table returns schema. Successors created inside a simulation start at ADR-0009.
Debt ledger	Seven open entries, including the dual write and the local refund-claim ledger — see the brownfield case.

How to score a response

There is no predetermined architecture answer in any of these eight cards, and a simulation that had one would be a quiz wearing a costume. Two responses can reach opposite conclusions and both score well; two responses can reach the same conclusion and one of them be worthless. What is being scored is the reasoning that connects evidence to decision to consequence.

Scoring criteria

Criterion	What earns credit	What loses it
Coherent adaptation	The change propagates. If a driver's measure moves, the test that proved it moves, the rollout gate that used it moves, and the forecast that assumed it moves. Nothing is left contradicting anything else.	A new design with the old plan still attached; a scope cut with no forecast change; an updated ADR whose consequences nobody traced.
Transparent trade-offs	The cost is named, priced and owned. "We accept a 6-hour worst-case label delay for the pilot cohort; support will see it; Lena owns the alert" is a trade. It survives someone disagreeing with it.	A change presented as free. Optimism about a dependency you do not control. A cost that appears only in the engineering version of the message.
The outcome is preserved — or renegotiated in the open	Either the response still moves resolution time and contact rate, or it says plainly: "this evidence means we cannot hit −25% by 30 September; here are two things we can hit instead; the sponsor decides by Friday."	Quietly retargeting. Redefining the measure so the old number still looks achievable. Reporting output because the outcome stopped moving.
History stays readable	A superseding ADR that cites its predecessor; a frame version bump with a one-line changelog; a driver whose measure changed on a date, for a reason, at someone's decision.	Editing the original decision. Deleting the assumption that turned out false. A plan that has always said what it says now.
Proportionality	The size of the response matches the size and reliability of the evidence. One study with twelve participants and one production metric over fourteen days are not the same weight, and a strong answer says which it is treating as decisive.	Replanning the initiative on one anecdote. Ignoring fourteen days of production data because it is inconvenient.
The unasked question	Naming what the injection does not say, and what you would need before deciding — with a timebox. Some of these cards cannot be answered responsibly without buying one specific piece of evidence first.	Confident certainty on incomplete information. Also: paralysis — asking for evidence that will not arrive before the decision must be made.

† Use the capstone rubric for the numeric part. The mapping at the bottom of this page says which dimensions each simulation exercises, so a low score on one card points at a specific practice rather than a general feeling of inadequacy.

Simulation 1 · The vendor

The carrier cannot meet the latency or availability you designed against

Starting position

Week 5. ADR-0003 is accepted and built: label intent is persisted before the carrier is called, a worker retries, the customer sees pending label. Driver AV-1 promises 99% of labels within 15 minutes of carrier recovery; assumption A3 says the carrier can meet the target. The pilot is at 1% and the rollout contract requires "zero duplicates, p95 within target" to advance to 10%.

The injection

Email · Tuesday 09:14 · from the carrier's account manager

Subject: Re: label API performance — Q4 platform migration

Thanks for the data you sent. I've had confirmation from our platform team. The label endpoint is being migrated to the new fulfilment platform through Q4, and during that period we are not able to commit to the response times in the technical annexe. Our current advice for integrators is to treat label creation as asynchronous with a 4-hour target and to expect intermittent unavailability windows of up to 90 minutes, mostly overnight CET. The 15-minute recovery figure in your note is not something we can support, and I'd rather tell you now than have you discover it in November. Happy to set up a call with our platform lead.

Your own measurements · label endpoint, last 14 days

Metric	Designed for	Observed (14 d)	Worst day
Label creation p95	under 30 s	4 min 10 s	27 min
Availability	99.5%	98.1%	91.4% (a 104-minute window on the 9th)
Labels created within 15 min of recovery	99%	61%	18%
Requests accepted despite carrier failure	100%	100%	100%

Commit

Write your five responses now. Before you do, notice the last row of that table: something in your design is already working. A strong answer starts by saying which part of the plan the evidence vindicates, because that is what tells you how much of it to change.

Reveal: two coherent responses

Response A — Change the promise, not the mechanism

The call: change. ADR-0003's mechanism is not falsified — it is confirmed. Persisting intent is exactly why 100% of requests were still accepted through a 104-minute outage. What is falsified is A3 and the measure inside AV-1: the 15-minute recovery figure was taken from the carrier's contract annexe and was never the team's own observation. So the design stands and the promise changes: label creation becomes explicitly asynchronous with a 4-hour target, the pending state gets a visible, honest estimate in the customer interface, and support gets a queue view of labels older than 4 hours.

Artifacts superseded

AV-1 measure changed: "99% of labels within 15 minutes of recovery" → "99% within 4 hours of request; 100% of requests accepted during unavailability up to 120 minutes". Dated, with the 14-day measurement cited as the reason.
Assumption A3 marked false in the assumption map, with the evidence and the date. Not deleted.
ADR-0003 stands; a short note is appended linking the new AV-1 measure. No successor is needed — this is the case where the decision was right and its input was wrong.
Story map: slice 4 (status visibility) moves ahead of slice 5 (agent override), because a 4-hour label promise only works if the customer can see it.
Frame v0.4 → v0.5: adds "label creation is asynchronous with a 4-hour target" to constraints. The outcome numbers are untouched — resolution time is measured to the refund decision, not the label.
Forecast unchanged at 70%, 23–30 September; the added status work displaces the bulk-returns screen, which was already the first scope trade.

To the sponsor (3 sentences)

The carrier has told us they cannot meet the label speed in their own contract annexe during their Q4 platform migration, and our own two weeks of measurement agrees with them. This does not change what customers get — every request is still accepted and the refund decision still lands in under four hours — but the shipping label now arrives within four hours instead of fifteen minutes, and we are moving the status screen forward so customers can see that rather than wonder. No change to the date or the pilot outcome; the bulk-returns screen comes out of the pilot to pay for it.

To engineering (5 bullets)

AV-1's measure is now 4 hours, not 15 minutes. The recovery test and the OP-1 alert threshold both change today — an alert calibrated to a promise we no longer make will page you for a system that is behaving.
ADR-0003 is not superseded. Intent-then-worker is the reason the 104-minute outage on the 9th cost us nothing; do not let anyone use this as evidence that the async design was wrong.
Slice 4 moves up. The pending state needs a real estimate and a support queue view of labels older than 4 hours, with an owner.
Retry policy stays capped and idempotent. A slower carrier is not a reason to retry harder — that is the shape of INC-2481.
New ledger entry: "carrier async promise depends on a status screen we have not built yet" — interest is support contacts about missing labels; trigger is slice 4 shipping; owner Asha.

Response B — Pause the vendor decision and buy a second option

The call: pause the vendor commitment; continue the pilot. The reading here is that a vendor who withdraws a contracted number five weeks before a pilot, for a migration lasting a quarter, is a single point of failure the initiative should not carry alone — and the right response is not to redesign around one carrier's worst quarter but to stop treating the carrier as fixed. The pilot continues at 1% on the current integration, with an agent-generated label fallback for anything older than 4 hours, while a timeboxed spike evaluates a second carrier behind the existing port.

Artifacts superseded

Frame v0.4 → v0.5: "carrier contract is fixed" moves from constraints to assumptions, with a note that the vendor has withdrawn a committed figure. This is the substantive change — a constraint that a vendor can unilaterally weaken was never a constraint.
New ADR-0009, "Second carrier evaluated behind the existing label port", citing ADR-0003 as context rather than superseding it. The port from ADR-0003 is what makes this cheap; that is worth writing down.
Spike brief: one question — "can carrier B create a label for our three highest-volume return types, and at what p95, over 200 sandbox calls?" — 3 days, one engineer, harness discarded.
AV-1 measure changed as in Response A (the 15-minute figure is indefensible either way), but marked provisional pending carrier evaluation.
Forecast drops to 60% confident, 25 September – 4 October, because three engineer-days leave the critical path. The moving variable is still scope; the agent-fallback path is the thing that could come out.

To the sponsor (3 sentences)

Our carrier has withdrawn the label speed they committed to in their contract, for the whole of Q4, which tells us something more useful than the number itself: we built the pilot on a supplier promise that the supplier can change at will. The pilot continues on schedule and customers are unaffected — agents will produce a label by hand in the rare cases that stall — while we spend three engineer-days finding out whether a second carrier is viable behind the same interface. That pushes our confident finish from 23–30 September to 25 September – 4 October, and I would rather have that option than the four days back.

To engineering (5 bullets)

The spike has one question and a discard rule. Two hundred sandbox calls, three return types, p95 and duplicate behaviour under timeout. We are not integrating carrier B this quarter; we are pricing the option.
The label port from ADR-0003 is the asset here. If the spike shows the port leaks carrier-specific vocabulary, that is a finding worth more than the latency numbers.
Agent fallback for labels older than 4 hours: manual, deliberately unautomated, capped at 20/day. If it exceeds that, we stop the 10% stage rather than build tooling for a temporary path.
Forecast is now 60%, 25 September – 4 October. Say that number when anyone asks; do not average it up in conversation.
Nobody touches the retry policy. A second carrier is a sourcing answer, not a reliability answer, and doubling retries at carrier A while we evaluate carrier B would give us two problems.

Scoring notes · what a strong response makes explicit

That the 15-minute figure was a vendor claim, never a measurement of your own — and that the team's own 14-day data is the stronger evidence in both directions.
That ADR-0003's mechanism was vindicated, not falsified. Weak responses redesign the async pipeline because the vendor disappointed them.
Whether the label is on the critical path of the outcome. It is not: resolution time is measured to the refund decision. A response that lets the label delay the outcome has not read the frame.
The customer-visible consequence and who absorbs it — the status screen, the support queue view, the agent fallback cap.
That an alert threshold calibrated to a withdrawn promise must move with it, or on-call is paged for correct behaviour.
A distinction between the vendor performance problem and the vendor dependency problem. Response A solves the first and accepts the second; Response B prices the second and pays for it in forecast. Naming which one you are solving is most of the answer.

Simulation 2 · The deadline

The pilot deadline is shortened by four weeks

Starting position

Week 2. The frame fixes the pilot at 30 September; the delivery map says date, outcome, guardrails and team size are fixed and scope is the moving variable. Slices 1 and 2 are built; slices 3–5 are not. The migration is at step 2 of 8.

The injection

Message · Thursday 17:40 · from the VP of Customer Operations

"Board's moved the operations review to 2 September and returns is the headline item — it's the one thing we've told them we're fixing this year. I need the pilot live and showing numbers by then. I know that's four weeks earlier than we said. Tell me what you need. If it's people I can probably find you two contractors from the platform team's budget."

Commit

Write your five responses. Two traps are set in that message, and a strong answer disarms both explicitly.

Reveal: two coherent responses

Response A — Change the scope; hold the outcome and the guardrails

The call: change. The date is now fixed by something outside the initiative, so the moving variable does what it was declared for. The pilot ships on 2 September as slices 1, 2 and 4 only — request, decision, refund, status visibility — for a narrower cohort: signed-in domestic customers, unopened items, within 30 days, refund to original payment method, no self-service label. Labels stay on the existing agent path for the pilot. Every guardrail is held, unchanged, because a guardrail that moves under deadline pressure was never a guardrail.

Artifacts superseded

Frame v0.4 → v0.5: pilot date 30 September → 2 September; new non-goal "self-service label creation (deferred to the post-pilot slice)"; outcome and all four guardrails unchanged; changelog line naming the board review as the cause.
Story map: slice 3 (label) moves out of the pilot release; slice 4 (status) moves in. Slice 5 (agent override) was already out.
ADR-0003 stands — the intent-then-worker design is what makes deferring the customer-facing label cheap. Its consequences section gains a line: "the agent path remains the pilot's label mechanism".
Rollout contract: stages unchanged, observation windows shortened from 3 days to 2 at the 1% and 10% stages, with an explicit note that this is the cost being paid and that the reversal criteria are not relaxed.
Forecast: 65% confident of 2 September for the reduced scope; the moving variable is now the cohort size, not the scope — if the migration's drift gate has not cleared by 26 August, the pilot runs on the legacy path with the new interface and 10% of the intended volume.

To the sponsor (3 sentences)

We can have a pilot live and producing real numbers by 2 September, but not the pilot we described: it will cover the refund decision and customer status, and the shipping label will still be produced by an agent as it is today. That still tests the thing the board cares about — whether customers resolve returns in hours instead of days — because resolution time is measured to the refund decision, and we hold every guardrail including fraud and the manual queue. I do not want the two contractors: adding people to a four-week window costs us more in explaining the system than it returns, and the honest constraint is our test coverage, not our headcount.

To engineering (5 bullets)

Slice 3 is out of the pilot. Do not half-build it. A label path that exists behind a flag but is never exercised is a maintenance cost with no evidence attached.
Observation windows go from 3 days to 2 at 1% and 10%. That is the cost of the date, and it is the only gate we are moving. The thresholds — zero duplicates, fraud flat, queue under 50 — do not move.
The seven-clean-days drift gate before the read switch does not move either. If it has not cleared by 26 August, the pilot runs on the legacy backend behind the new interface. Plan for that branch now; do not discover it on the 26th.
No contractors. If anyone offers again, the answer is the same and the reason is the same: four weeks is shorter than the time to make someone useful in this codebase.
New ledger entry: "shortened observation windows at 1% and 10%" — interest is a lower chance of catching a slow-moving guardrail regression; trigger is the first post-pilot stage, where the windows go back to 3 days; owner Lena.

Response B — Renegotiate what "the pilot" means

The call: change, by redefining the deliverable rather than the scope. The reading here is that the board does not need the pilot; it needs credible evidence on 2 September. Those are different products. A 1% exposure on 2 September gives roughly 60 returns — too few to say anything about a 25% contact-rate change with any confidence, so shipping a thin pilot on time would produce a number that is not evidence and would then be quoted for a year. Instead: ship the full first slice to staff and a 200-customer invited cohort on 2 September, present it as an early read with stated uncertainty, and keep 30 September as the date the pilot outcome is actually called.

Artifacts superseded

Frame v0.4 → v0.5: splits one date into two — "early read, 2 September, invited cohort of 200" and "pilot outcome call, 30 September, eligible domestic cohort". The outcome targets attach to the second date only, explicitly.
Assumption map: new entry A6, "a 200-customer cohort produces a directional read on resolution time but not on contact rate", with the sample-size reasoning attached — so nobody later treats the September 2nd number as the result.
Rollout contract: an extra stage inserted between staff and 1% — "invited cohort, 200 customers, 2 weeks, opt-in, support briefed" — with its own entry and reversal criteria.
Story map unchanged. This response deliberately cuts nothing, which is its main risk and should be stated as such.
Forecast: 70% confident of the early read on 2 September; 55% confident of the full pilot by 30 September, down from 70%, because the invited cohort's support and analysis load lands on the same four people.

To the sponsor (3 sentences)

We can have real customers using this on 2 September and I can stand up at the review with their numbers — but I want to be straight that 200 customers over two weeks will tell us whether resolution time is dropping and will not tell us whether support contacts fall by 25%, because that needs several times the volume before the number means anything. So I would like to present it as an early read with the uncertainty stated, and keep 30 September as the date we call the pilot result. The risk I am taking is that we are not cutting scope to buy this, so my confidence in the 30 September date drops from 70% to 55%, and I will tell you in two weeks if that needs to change.

To engineering (5 bullets)

New rollout stage: invited cohort of 200, opt-in, two weeks, its own entry and reversal criteria. It is a stage, not a demo — same gates, same telemetry, same on-call.
Nothing comes out of scope. That is a real risk and you should push back on it if week 3 slips; the first thing to cut is still slice 3.
Instrument the early read so its sample size is visible in the dashboard itself. If a number is going in front of a board, the confidence interval goes with it, in the same screen.
The drift gate and the guardrail thresholds are untouched. An invited cohort is not a reason to lower them; if anything it is 200 people who agreed to help us and will notice.
Forecast for 30 September is now 55%. It is lower than Response A's because we bought no scope relief — that is the trade, and it is on us to say so before someone else notices.

Scoring notes · what a strong response makes explicit

The offer of contractors is declined, with a reason. Brooks' point is not that people never help; it is that four weeks is shorter than the ramp, and the offer's real cost is the two weeks of an existing engineer's attention. Accepting silently is a weak answer; declining without a reason is nearly as weak.
The guardrails do not move. A response that trades fraud tolerance or the accessibility standard for four weeks has traded the wrong variable, and has done it where the executive cannot see it.
Which measurement the shortened date can and cannot support. A pilot that produces a number too small to mean anything is worse than no number, because the number will be quoted.
What comes out of scope, by name, and what stays — plus the second thing that would come out if it slips again. Naming the next cut is what makes the first one credible.
A new forecast with a confidence and a moving variable, stated to the sponsor rather than only in the plan.
Which gate is being relaxed (observation window) and which is not (thresholds, drift). Conflating those two is the most common way a shortened deadline turns into an incident.

Simulation 3 · Security

Security identifies a new trust boundary

Starting position

Week 4. Slice 1 shipped in week 3 and includes photo evidence upload for damaged-item returns — added during discovery because agents said it would cut the manual queue. Driver SE-1 covers cross-customer disclosure of order and return data. Evidence images are stored in the existing media bucket and served from the CDN by unguessable URL, the same mechanism product photos use. The pilot is at 1% and due to reach 10% on Monday.

The injection

Security review finding · Thursday · from the platform security engineer

Field	Finding
Title	Return evidence images are served without object-level authorization
Severity	High. Not currently exploited; no evidence of access outside expected sessions in 30 days of CDN logs.
Detail	Evidence images inherit the product-media path: the CDN serves any object to any requester holding the URL, with no session check. Product photos are public by design; return evidence is not. Images have contained shipping labels with home addresses, a prescription box, and in two cases a face.
Trust boundary	The CDN edge is a boundary the returns design never modelled. The threat model covers the application's API surface and the partner API; it does not mention media delivery. The warehouse app also renders these URLs into its picking screens, which are accessible to 140 warehouse staff and 3 third-party logistics contractors.
Ask	"Tell me your decision by Monday. I am not going to block your rollout — that is your call and your data — but I want the decision written down and I want to know who made it."

Commit

Write your five responses. Note the last row: you have been handed the decision rather than a mandate, which is a more demanding position than being blocked.

Reveal: two coherent responses

Response A — Change the design; hold the 10% stage for one week

The call: change, and pause exposure while the change lands. The finding is not that a control failed; it is that a trust boundary was never in the model, which means the threat model — not just the code — is wrong. Evidence images move behind the application: short-lived signed URLs issued per request after an object-level authorization check that the return belongs to the requesting customer, or that the requester is an agent with a case assignment. The warehouse app gets its own short-lived URLs through a scoped endpoint, and the third-party contractors are excluded from evidence entirely, which is a policy decision made in the open rather than a side effect of a CDN path.

Artifacts superseded

SE-1's scenario rewritten to name the boundary: "if any actor holds an evidence object URL without an authorized session, disclose nothing, in every environment" — with negative acceptance examples added to the example set (expired URL, another customer's return, an unassigned agent, a contractor account).
New ADR-0010, "Return evidence served via short-lived signed URLs with object-level authorization", superseding the implicit decision inside slice 1's design note. It records the rejected option — a private bucket with an application proxy — and why: the proxy would put image bytes on the request path that PF-1 measures.
C4 container view updated: the CDN appears as a component with a trust boundary drawn around it, because a boundary that is not on the picture is a boundary nobody reviews.
Threat model: new entry for media delivery, and a standing question added to the review checklist — "does this feature create any object served outside the application's authorization path?"
Rollout contract: the 10% stage is held for one week; a new entry condition is added — "negative authorization examples pass in the pilot environment".
Forecast: 70% → 65%, 25 September – 2 October. Four engineer-days move; the bulk-returns screen comes out, as previously signalled.

To the sponsor (3 sentences)

Our security review found that the photos customers upload with damaged-item returns are served the same way as public product images, which means anyone holding the link can open them — and some of those photos contain home addresses and, in two cases, a person's face. There is no sign anyone has done this, but I am not willing to increase exposure from 1% to 10% on Monday while it is true, so we are holding that step for one week while we put the images behind a proper permission check. The pilot date moves from a 23–30 September window to 25 September – 2 October, and the bulk-returns screen comes out of scope to limit that.

To engineering (5 bullets)

Evidence images move to signed, short-lived, per-request URLs with an object-level check. Not a secret path, not a longer random token — an authorization decision, made per request, by the application.
The negative examples are the deliverable, not the fix. Expired URL, another customer's return, an unassigned agent, a contractor account: four tests that must fail closed, in CI, before the 10% gate opens.
The warehouse app gets a scoped endpoint. Do not hand them a longer-lived token because it is easier — 140 staff and 3 contractor accounts is exactly the population this control exists for.
The 30-day CDN log review is evidence we were lucky, not evidence we were safe. Say it that way if anyone summarizes this upward.
Add "does this create an object served outside the app's authorization path?" to the design review checklist. This class of finding is cheap to catch and expensive to discover.

Response B — Remove the feature from the pilot

The call: pause the capability, continue the pilot. The reading here is that photo evidence entered scope during discovery on an agent's assertion and has never been tested against the outcome. It carries the highest-sensitivity data in the initiative — addresses, a prescription, a face — for a benefit nobody has measured. Rather than spend four engineer-days building an authorization mechanism for an unproven feature four weeks before a pilot, turn evidence upload off, route damaged-item returns to the existing agent flow, and decide after the pilot whether the manual-queue saving justifies holding that data at all.

Artifacts superseded

Frame v0.4 → v0.5: "customer-uploaded evidence images" moves to non-goals for the pilot, with the security finding cited as the reason. The manual-queue guardrail (≤ 50/day) is unchanged and is now the thing to watch, since evidence upload was meant to reduce it.
New ADR-0010, "Defer customer-uploaded return evidence", recording the rejected option (build signed URLs now) and the accepted cost: the manual queue may rise, and the decision must be revisited with that number in hand.
Assumption map: new entry A7, "photo evidence reduces the manual review queue", marked untested — the feature that would have tested it is deferred. This is the honest version; deleting the feature and the assumption together would erase the question.
Data deletion: the 41 images already uploaded during the 1% stage are deleted within 7 days, with the deletion recorded. Deferring a feature does not defer the data it already collected.
SE-1 and the threat model still gain the CDN boundary entry — the boundary exists whether or not this feature does, and the warehouse app's picking screens still render product media through it.
Forecast unchanged at 70%, 23–30 September, and this is the response's main argument.

To the sponsor (3 sentences)

Security found that customer-uploaded return photos are served without a permission check, and those photos have included home addresses and people's faces, so we are switching the upload off today and deleting the 41 images already collected. That feature was added on our agents' recommendation and we have never actually measured whether it reduces their workload, so rather than spend four engineer-days securing something unproven, we will run the pilot without it and watch the manual review queue. The pilot date does not move, and if the queue climbs toward its 50-a-day limit we will have the evidence to justify building it properly afterwards.

To engineering (5 bullets)

Upload is off today behind the existing flag; damaged-item returns route to the agent queue. Do not leave a disabled upload control in the interface — the affordance is the promise.
Delete the 41 existing images within 7 days and record the deletion. This is not optional cleanup; it is the part of the response that actually reduces risk.
The CDN trust boundary still goes into the threat model and the container view. We removed a feature, not a boundary.
Manual queue is now the metric that decides whether this comes back. Instrument the damaged-item share of it specifically, or the post-pilot argument will be anecdote against anecdote.
If anyone asks for "just a longer random URL" as a compromise, the answer is no, and the reason is that unguessability is not authorization — write it in the ADR so it is answered once.

Scoring notes · what a strong response makes explicit

That the boundary, not the bug, is the finding. The threat model and the container view are updated in both responses, including the one that removes the feature. A response that only patches the URL scheme has fixed an instance and left the class.
The distinction between unguessability and authorization, stated in a form that answers the "just make the token longer" suggestion once and permanently.
What happens to data already collected. Deferring or securing a feature does nothing about the 41 images that exist right now, and a response that does not mention them has missed the only irreversible part of the situation.
Who the affected population actually is — 140 warehouse staff and 3 third-party contractors, not "internal users". Precision here is what turns a severity rating into a decision.
That the decision was delegated, and therefore has a named owner and a written record. Security explicitly declined to block; a response that behaves as if it were blocked has avoided the responsibility rather than exercising it.
Whether the feature has ever been justified against the outcome. Response B's strength is asking a question Response A never asks; Response A's strength is that it does not lose a capability agents asked for. Naming which risk you prefer is the judgment.

Simulation 4 · Research

Research disproves the primary usability assumption

Starting position

Week 5. Assumption A1 — "customers will self-serve" — is the load-bearing belief of the whole initiative, and A4 says status visibility is secondary to decision speed. The story map puts status visibility at slice 4, after refund and label. The pilot is at 1%, and the outcome measured so far is median resolution 3.4 hours (target: under 4) with support contacts down 6% (target: 25%).

The injection

Research readout · Friday · from the designer, after 12 moderated sessions and 190 pilot-cohort session recordings

Finding	Evidence	Strength
Customers complete the request flow easily	11 of 12 moderated participants completed unaided, median 2 min 40 s	Strong
They contact support anyway — after a successful submission	Of 190 recorded pilot sessions, 58 submitted successfully and then contacted support within 24 h. Contact rate among submitters is 31%, against 41% baseline.	Strong
The reason is not decision speed. It is not knowing what happens next.	All 12 moderated participants asked an unprompted "so what happens now?" question. In 9 of 12, the phrase used was about the refund, not the return.	Moderate — small n, and the moderator's prompt may have led it
"Approved" is read as "money is coming back today"	7 of 12 expected the refund same-day; the actual bank settlement is 3–5 working days and is not stated anywhere in the flow	Moderate
Decision speed itself is barely noticed	No participant commented on the 4-hour target; 3 assumed the decision was instant and were surprised it was not	Weak — not what the sessions were designed to test

Commit

Write your five responses. This is the card where proportionality is scored hardest: the findings above have three different evidence strengths and a strong answer treats them differently.

Reveal: two coherent responses

Response A — Change the intervention; the opportunity stands

The call: change. This is lesson 0050's middle option in its purest form: the opportunity — customers waiting and then calling — is confirmed, and the mechanism the team chose to address it is wrong. A1 survives; customers self-serve happily. A4 is falsified: status visibility is not secondary, it is the whole contact-rate story, and the 6%-against-25% gap is explained by building the fast decision and not the visible aftermath. Slice 4 moves to the front, and it grows: expected refund settlement dates, a state the customer can return to, and a proactive notification when the refund actually settles.

Artifacts superseded

Assumption map: A4 marked false with date and evidence; A1 marked confirmed. New A8: "customers contact support because the refund timeline is invisible, not because the decision is slow" — marked moderate confidence, being tested by slice 4.
Frame v0.4 → v0.5: the outcome numbers are unchanged, but the problem statement is rewritten. It previously said returns wait 72 hours for a decision; it now says returns wait 72 hours for a decision and customers cannot see what happens after it. Changing the problem statement rather than the target is the substantive move.
Story map: slice 4 becomes slice 2. Slice 3 (label) and slice 5 (agent override) both move back. The refund slice keeps its position.
New ADR-0011, "Expose refund settlement timeline as first-class state", superseding nothing but recording the consequence: settlement dates come from the payments team's API, whose availability now sits on a customer-facing path it was not on before. Fallback: show a range, never a false precision.
MO-1 unchanged; PF-1 unchanged; a new outcome telemetry cut is added — contact rate by time since submission, which is the segmentation that would have found this in week 3.
Forecast: 70% → 68%, window unchanged. Slice 4 was already scoped; this is a reordering, not new work, and saying so protects the response from looking like a bigger change than it is.

To the sponsor (3 sentences)

Customers are using the new flow easily and getting a decision in about three hours, so the core bet is working — but they are still calling us, and research shows it is because after we say "approved" they have no idea when their money arrives, and most of them assume it is today. That means the biggest remaining lever on the support-contact target is not making the decision faster, it is making the refund timeline visible, so we are moving that work from fourth in the plan to second and adding the settlement dates. The date and the targets do not change; what changes is the order, and I would rather tell you the plan moved than report a 6% improvement against a 25% target for another three weeks.

To engineering (5 bullets)

Slice 4 becomes slice 2. Slice 3 and 5 move back. This is a reorder of already-scoped work, not new scope — hold that line if it gets described as a pivot.
Settlement dates come from the payments API, which puts a dependency we do not own on a customer-facing path for the first time. Cache it, show a range not a date, and never show a precise date we cannot honour.
New telemetry cut today: contact rate by time since submission. It is one dimension and it is the one that would have caught this in week 3.
A4 is marked false in the assumption map, not deleted, and A1 is marked confirmed. Both matter — the next person to read this will otherwise conclude the whole premise was wrong.
The "3 assumed the decision was instant" finding is weak evidence from sessions not designed to test it. Do not build anything on it. If it matters, it is a question for the next study.

Response B — Pause the reorder; test the finding first

The call: pause the replan; continue the build. The reading here is that the decisive claim — why customers contact support — rests on 12 moderated sessions with a possible moderator prompt, flagged "moderate" by the designer herself, while the strong evidence (31% contact rate among submitters) says only that they contact, not why. Reordering the story map on moderate evidence is a two-week commitment made on a small sample. Instead: buy the missing evidence in five days — code the actual support-contact reasons for the 58 pilot contacts, which already exist in the ticket system — and hold the reorder until then. Slices continue as planned in the meantime.

Artifacts superseded

Assumption map: A4 marked challenged, under test rather than false, with the specific test named and dated. A1 marked confirmed. The distinction between challenged and false is this response's whole argument.
Spike brief: one question — "of the 58 pilot support contacts, what share are about refund timing?" — five days, the designer plus a support lead, output is a coded list, harness discarded. Decision rule written in advance: above 50%, the reorder happens; below 25%, it does not; between, one more week of tickets.
Story map unchanged this week. Explicitly: this is a decision to spend a week not deciding, and it is only defensible because the evidence arrives inside the pilot window.
New telemetry cut added immediately, as in Response A — contact rate by time since submission — because it costs a day and improves both branches.
Forecast unchanged at 70%, with a stated conditional: if the reorder happens after the readout, slice 3 leaves the pilot and the window moves to 25 September – 2 October.

To the sponsor (3 sentences)

Research suggests customers are calling us because they cannot see when their refund arrives rather than because the decision is slow, and if that is right it changes what we build next — but the finding rests on twelve interviews and our designer has flagged it as moderate confidence, so I do not want to reorder three weeks of work on it yet. We already have 58 real support contacts from the pilot sitting in the ticket system, and we can code them in five days to find out directly; I have written down in advance what result would change the plan and what result would not. Nothing slips this week, and you will have the answer and my decision next Friday.

To engineering (5 bullets)

No reorder yet. Keep building slice 3. If you are told the plan is changing, it is not — it is being tested.
The decision rule is written before the data: above 50% refund-timing contacts we reorder, below 25% we do not, in between we take another week. Nobody negotiates the threshold after seeing the number.
Add the contact-rate-by-time-since-submission cut today anyway. It is a day of work and it is useful whichever way the readout goes.
A4 is "challenged, under test", not false. Precision in the assumption map is what lets us reopen this without relitigating it.
If the readout says refund timing, slice 3 leaves the pilot and the window moves to 25 September – 2 October. Know that branch now so we are not designing it under time pressure next Friday.

Scoring notes · what a strong response makes explicit

Which findings are strong and which are not. The 31% contact rate among submitters is production data; the reason behind it is twelve interviews with a flagged confound; the "decision speed is barely noticed" line is not evidence at all. A response that treats all five rows equally has failed the proportionality criterion regardless of what it decides.
That A1 is confirmed, not disproved. The card is titled "research disproves the primary usability assumption" and the honest reading is that it disproves a different one — A4. Noticing that the headline is wrong is the finding.
That the problem statement changes even though the outcome targets do not. The frame's description of the problem was incomplete; the targets were fine.
A telemetry change that would have surfaced this earlier — contact rate segmented by time since submission — and an acknowledgement that its absence is why this arrived in week 5 rather than week 3.
A decision rule set before the evidence arrives, if the response buys evidence. A spike with no pre-agreed threshold becomes a negotiation about the number.
That the new dependency is named: a customer-visible settlement date comes from another team's API, and showing a precise date you cannot honour would convert a support contact into a complaint.

Simulation 5 · The regulator

A regulator changes the data-retention requirement

Starting position

Week 6. The frame's constraints include "PII is retained for 30 days" and "no payment data enters the new module". The returns module stores, per return: customer identifiers, the delivery address on the label, free-text reason notes, and (in the variant where it survived simulation 3) evidence images. A deletion job runs nightly at day 31. The pilot is at 10% and due to reach 50% next Wednesday.

The injection

Note · Monday 11:20 · from Legal, forwarded from the consumer-protection regulator's updated guidance

"Two changes take effect on 1 January and they pull in opposite directions, which is why I want engineering in the room rather than a written answer.

One: records substantiating a refund or refusal decision must be retained for 24 months and be producible on request within 10 working days — that is the decision, the reason, the amount, the date, and any evidence relied on. Our current 30-day deletion would put us in breach the day it applies.

Two: a customer's erasure request must be honoured within 30 days, and the guidance narrows the grounds for refusing it. The overlap between these two is real and the regulator knows it: the resolution is that the substantiating record is retained, but reduced to what actually substantiates the decision.

I need to know by the end of next week what you would have to retain, in what form, and whether that is compatible with erasure. It applies to the whole of returns, not just your pilot."

Commit

Write your five responses. The word to notice in that note is substantiating: it is doing more design work than any sentence in the injection.

Reveal: two coherent responses

Response A — Change the data model: split the decision record from the person

The call: change. The two requirements only conflict if the substantiating record and the personal data are the same record. They are not, and separating them is a domain modelling job, not a compliance one. The decision record — return id, order reference, decision, reason code, amount, date, the policy version applied, and a hash of any evidence relied on — is retained 24 months and contains no direct identifiers. The personal layer — name, address, free text, images — hangs off it, is retained 30 days by default, and is erasable independently. An erasure request removes the personal layer and leaves a decision record that still substantiates what was decided and why, which is exactly what the regulator asked for.

Artifacts superseded

Frame v0.5 → v0.6: the constraint "PII retained for 30 days" is replaced by two constraints — "decision records retained 24 months, no direct identifiers" and "personal layer retained 30 days, erasable within 30 days of request" — with the guidance date and Legal named as the source.
New ADR-0012, "Separate substantiating decision records from the personal layer", superseding ADR-0007 (the original single-table returns schema decision), which gains status superseded by ADR-0012 and keeps its text. Rejected options recorded: retain everything 24 months (fails erasure); pseudonymize in place (the reason text and the address on a label re-identify trivially).
Data model: the returns table splits into return_decisions (24 months, no identifiers, immutable) and return_details (30 days, erasable). This is a second expand/contract sequence, deliberately started after the pilot — the first one is not finished, and running two overlapping migrations on orders is how the lock discipline gets broken.
New driver PR-1 (privacy): "when a customer requests erasure, remove every direct identifier within 30 days while leaving the decision record intact and producible" — with an evidence mechanism: an automated test that performs an erasure and then produces the substantiating record.
Debt ledger: the nightly 30-day deletion job is now wrong for one of the two tables. Interest is a compliance exposure that begins 1 January; trigger is the second migration's read switch; owner Asha with Legal as the reviewer.
Forecast for the pilot unchanged — 1 January is fifteen weeks away and putting this work into the pilot window would be the wrong response to a deadline that is not urgent yet. That reasoning is stated rather than left implicit.

To the sponsor (3 sentences)

The regulator is asking us to keep the record of every refund decision for two years while still deleting a customer's personal data on request within thirty days, and those only conflict if we store both in the same place — which we currently do. The fix is to split them: the decision, the reason and the amount live for two years with no name or address attached, and the personal details sit separately and can be erased without destroying our ability to explain the decision. This does not affect the pilot or its date; the change applies from 1 January and I am proposing we do it in the two months after the pilot, with Legal reviewing the split before we build it.

To engineering (5 bullets)

Two tables, two lifetimes: return_decisions is immutable and identifier-free for 24 months; return_details is erasable and expires at 30 days. The join key is the return id, which is ours, not the customer's.
"No direct identifiers" includes the free-text reason and the shipping address on a label image. If it re-identifies, it belongs in the personal layer — assume it does and argue the other way with evidence.
This is a second expand/contract on the same tables, and it does not start until the first one reaches step 7. Two overlapping migrations on orders is how INC-2503 happens again.
New driver PR-1 with a real test: erase a customer, then produce their substantiating decision record. If that test cannot be written, the model is wrong.
The nightly deletion job is now incorrect for one table. It is in the ledger with an owner and a trigger — do not "fix" it ad hoc before the split exists, or you will delete the records we are being asked to keep.

Response B — Stop collecting what you cannot justify keeping

The call: change, by minimizing rather than restructuring. The reading here is that the cheapest compliant system is the one holding the least data, and the guidance is an opportunity to ask a question the initiative skipped: what does the module actually need? Free-text reason notes are used by nobody downstream — the reason code drives everything — and the delivery address is already authoritative in orders, which has its own retention. So: stop storing free text, stop copying the address, keep evidence images only as a hash plus the agent's finding, and the 24-month record becomes small enough that erasure barely touches it.

Artifacts superseded

Frame v0.5 → v0.6: same two retention constraints as Response A, plus a new one — "the returns module stores no personal data that exists authoritatively elsewhere".
New ADR-0012, "Minimize returns data rather than partition it", superseding ADR-0007. Rejected option recorded: the two-table split, rejected here as solving a storage problem the team could avoid having. Accepted cost stated plainly: free-text reasons are lost, and if agents later need them the decision must be reopened — with a named revisit trigger.
Story map: a small slice is added inside the pilot — replacing the free-text field with a structured "other" reason and an optional agent note that is not retained past 30 days. It is roughly two days and it is inside the pilot because collecting less is only cheap before you have collected it.
New driver PR-1 as in Response A, with an additional fitness check: a test that asserts the returns schema contains no column carrying a direct identifier, run in CI. A privacy property that is not checked is a privacy intention.
Assumption map: new A9, "free-text return reasons are not used by agents or finance", marked believed, tested by asking both before the change lands — because this response's whole case rests on it and it has not been verified.
Forecast: 70% → 68%, window unchanged. Two days of work inside the pilot, paid for from the same buffer as before.

To the sponsor (3 sentences)

The regulator wants two years of records substantiating each refund decision and thirty-day deletion of personal data on request, and the simplest way to satisfy both is to stop holding personal data we do not need: the address already lives in the order system, and nobody actually reads the free-text reason customers type. We are proposing to store a reason code and the decision rather than free text, which makes the two-year record small, impersonal and cheap to keep, and makes erasure almost trivial. It is about two days of work inside the pilot rather than two months after it, but I want to check with our agents and with finance first that nobody is quietly relying on that free text.

To engineering (5 bullets)

Stop copying the delivery address into returns; reference the order. Duplicated personal data is duplicated retention obligation.
Free text becomes a structured reason plus an optional agent note with a 30-day life. Ask agents and finance before it lands — A9 is a belief, not a finding, and it is the assumption this whole response rests on.
Evidence images reduce to a hash plus the agent's recorded finding. The hash substantiates that the image existed and was relied on; the image itself does not need to survive 24 months.
New CI check: no column in the returns schema carries a direct identifier. Write it now, while the schema is small enough for the check to pass.
Two days inside the pilot, on purpose. Every week we wait is another week of data we then have to migrate and delete — collecting less is only cheap before you have collected it.

Scoring notes · what a strong response makes explicit

That the two requirements do not actually conflict once you separate the substantiating record from the person. A response that presents this as an unresolvable tension, or that asks Legal to choose, has not done the modelling the note explicitly asked engineering to do.
What "substantiating" means, enumerated: decision, reason, amount, date, policy version, and a reference to any evidence relied on. Everything not on that list is a candidate for the erasable layer.
That free text and a shipping address re-identify a person, so pseudonymizing in place is not a real option. Naming the rejected option and why is what makes the chosen one credible.
The interaction with the migration already in flight. Starting a second expand/contract on orders during the first one is the single most dangerous thing available in this card, and a strong response either sequences them or says explicitly why it can overlap.
Whether the work belongs inside the pilot window. Both reveals are defensible and they disagree: Response A defers because 1 January is fifteen weeks away, Response B pulls two days in because collecting less is cheapest before the data exists. Either is fine with the reasoning attached; neither is fine by default.
A privacy driver with an executable check — an erasure-then-produce test, or a schema assertion in CI. A retention rule enforced by a nightly job nobody tests is the compliance equivalent of a comment.

Simulation 6 · The dependency

Another team cannot deliver its dependency

Starting position

Week 3. Refunds go through the payments team's POST /refunds (v2), which has no idempotency key; the returns team's compensating control is the local refund-claim wrap plus a daily reconciliation against the payments ledger. The payments team's v3, with per-request idempotency keys, is committed for 14 November — after the pilot, and the plan already assumes it will not arrive in time. The reconciliation feed is a nightly CSV export the payments team produces.

The injection

Message · Wednesday 08:05 · from the payments platform lead

"Heads up before you hear it elsewhere. We've been pulled into the PCI re-certification and it's taking the whole team until mid-November. Two consequences for you.

First, v3 slips from 14 November to Q2 next year. I know you weren't depending on it, which I appreciate.

Second, and this is the one I'm sorry about: we're freezing all changes to the v2 refund path from Friday until certification completes, and the auditors have flagged the nightly CSV export as an uncontrolled data flow. We have to turn it off. I can't give you a replacement inside the freeze — anything new on that path is a change. Realistically you get something in December.

Ask me for what you need and I'll see what's already built, but I can't promise new work."

Commit

Write your five responses. The slip is the headline and the second paragraph is the actual problem — a strong answer says so in its first line.

Reveal: two coherent responses

Response A — Change the control; make our own record authoritative

The call: change. The v3 slip changes nothing, because the plan never assumed it — that is what the fallback was for, and it should be said out loud so the team gets the benefit of a decision it already made well. What breaks is the detection side: the reconciliation feed was how the team would learn that a duplicate refund had happened despite the wrap, and losing it turns a controlled risk back into INC-2481's exact shape, where finance found the problem three days later. So the local RefundClaim ledger stops being a stopgap and becomes the authoritative record of refund intent, with detection moved to signals the returns team owns: a claim-state audit that alerts on any claim without a terminal state after 15 minutes, and a weekly manual spot-check of 50 claims against the payments admin interface, which is read-only and therefore not a change to the frozen path.

Artifacts superseded

New ADR-0013, "Local refund claim ledger is authoritative for refund intent", superseding ADR-0006 ("reconcile refunds against the payments export nightly"), which keeps its text and gains superseded by ADR-0013. Consequences recorded: we now own a record that must not be lost, so it gets a backup and restore rehearsal it did not previously need.
OP-1 gains a second scenario: "if a refund claim has no terminal state 15 minutes after creation, on-call is alerted and can identify the affected customer within 10 minutes" — with the alert as the evidence mechanism.
Debt ledger: the "local refund-claim ledger compensating for payments v2" entry has its trigger rewritten. It was "v3 in production, or a customer-visible duplicate"; v3 is now Q2, so the trigger becomes "a customer-visible duplicate, or Q2 whichever is first", and the interest rises from 2 hours to about 5 per month because of the weekly spot-check.
Dependency map: payments moves from fallback in force, evidence expected November to fallback is now the design; no evidence expected before Q2. The escalation trigger is unchanged and is still an event, not a date.
Forecast unchanged. The spot-check is an hour a week of an existing person, not new build.

To the sponsor (3 sentences)

Payments have been pulled into a PCI re-certification, so the improvement we were expecting from them in November now lands in the spring — which does not affect us, because we deliberately built our own safeguard rather than waiting for theirs. What does affect us is that they have to switch off the nightly reconciliation file we used as a second check, so we are making our own refund record the authoritative one and adding an alert that fires within fifteen minutes if a refund is ever left in an unknown state. That costs about an hour a week of manual checking until they can give us a replacement in December, the pilot date does not move, and I have written down that a duplicate refund reaching a customer escalates to both engineering managers the same day.

To engineering (5 bullets)

The v3 slip costs us nothing and that is not luck — the wrap was built for exactly this. Say it that way; teams that do not notice their own good decisions stop making them.
Losing the CSV is the real change. Our claim ledger is now authoritative, which means it needs a backup and a rehearsed restore. Something we merely reconciled against became something we cannot lose.
New alert: any claim without a terminal state after 15 minutes. Threshold is one. This replaces a nightly file with a fifteen-minute signal, which is strictly better and should be described that way.
The weekly spot-check of 50 claims uses the payments admin UI, read-only. Do not build tooling against a frozen path; do not ask them for an API during a freeze.
Ledger updated: trigger moves to Q2, interest rises to ~5 hours a month. If that number climbs again, it becomes an escalation rather than a habit.

Response B — Pause the refund slice; ship decision and status

The call: pause the automated refund path for the pilot. The reading here is that the team is now issuing money through a frozen, non-idempotent API with no independent reconciliation, defended by a control it wrote itself three weeks ago and has never seen fail. That is a reasonable risk with a second pair of eyes on it and a less reasonable one without. So the pilot ships slices 1 and 4 — request, decision, status visibility — and the refund itself stays on the existing agent path until the reconciliation feed returns in December. The outcome measure survives: median resolution is measured to the decision, and the agent refund step is the same one customers get today, so nothing gets slower.

Artifacts superseded

Frame v0.4 → v0.5: "refund to original payment method, automatically" moves from the first slice to a post-pilot slice, with the payments freeze cited. The outcome and guardrails are unchanged; a note records that the resolution-time measure ends at the decision, so the outcome remains testable.
Story map: slice 2 (refund) leaves the pilot release; slice 4 (status) moves into it. Slice 3 (label) was already at risk and now leaves as well — this response cuts more than it strictly must, and says so.
ADR-0006 is not superseded; it is marked suspended, pending the payments freeze, with the resumption condition written down. Suspension is the honest status when the decision is still right and its precondition has vanished.
Debt ledger: the refund-claim entry is closed for the pilot's duration and reopened as "agent refund path carries the pilot's refund volume" — interest is agent time, roughly 20 minutes per 50 returns; trigger is the reconciliation feed returning.
Forecast: 70% → 75%, window 20–27 September. This is the only response in these eight simulations where confidence goes up, and the reason should be stated: removing the highest-variance dependency from the pilot is what bought it.

To the sponsor (3 sentences)

The payments team has to freeze their refund system for a compliance audit and switch off the daily file we used to check that no refund was ever issued twice — and issuing refunds automatically against a frozen system with no independent check is not a risk I want to take with real customers' money in a pilot. So we will ship the fast decision and the status visibility, which is where the customer benefit and both our targets actually come from, and leave the refund itself on today's agent path until their file comes back in December. That takes a whole class of risk out of the pilot, moves my confident window slightly earlier to 20–27 September, and costs our agents about twenty minutes per fifty returns.

To engineering (5 bullets)

Automated refunds come out of the pilot. The claim wrap stays in the codebase, tested and unused — do not delete it, we are resuming in December.
ADR-0006 is suspended, not superseded. Write the resumption condition into it: the reconciliation feed, or an equivalent independent check, exists again.
Slice 3 leaves too. If we are cutting, cut once and cleanly rather than discovering the second cut in week 6.
Resolution time is still measured to the decision, so the outcome is still testable. Check the dashboard's definition matches that today — if it measures to refund settlement, the number will look worse for a reason that has nothing to do with us.
Forecast goes up, to 75% and 20–27 September. Explain it as removing variance, not as gaining time, or the next person to cut scope will expect the same gift.

Scoring notes · what a strong response makes explicit

That the slip is not the problem. The plan never depended on v3; the fallback was built precisely for this. A response that treats the slip as the emergency has not read its own dependency map — and misses that the second paragraph removes a control.
The difference between a control and a detection mechanism. The wrap prevents duplicates; the CSV detected them. Losing detection while keeping prevention is a specific, nameable degradation, and INC-2481 is the evidence for why it matters.
That a record you merely reconciled against becomes a record you cannot lose once it is authoritative — so backup and restore rehearsal enter scope with the decision, not later.
Respect for the other team's constraint. Neither response asks payments to break a freeze or escalates to move a compliance deadline; both find something already built, read-only, or internal. Spending influence where it cannot work is a lead-level error.
Whether the ADR is superseded or suspended. They are different states with different resumption conditions, and choosing the right one is the artifact-hygiene point of this card.
An updated debt-ledger trigger. The old trigger referenced a date that no longer exists, and a trigger pointing at a cancelled event is worse than no trigger — it looks handled.

Simulation 7 · Volume

Production volume is ten times the forecast

Starting position

Week 7. Assumption A5 puts pilot volume around 40 rps at peak; driver PF-1 specifies eligibility p95 under 800 ms at 40 rps, and that is the load the team tested. The pilot is at 50%, the manual review queue guardrail is 50 cases per day, and the dual write is still running, so every return costs two writes.

The injection

Metrics · Monday 07:00 – 11:00, after a partner enabled self-service returns for all its sellers on Sunday night

Metric	Designed / tested	07:00	09:00	11:00
Returns requests, peak rps	40	96	310	412
Eligibility p95	under 800 ms	740 ms	1.9 s	4.4 s
Database primary CPU	—	41%	78%	94%
Replica lag	under 2 s	0.4 s	3.1 s	11 s
Manual review queue, cases added	≤ 50/day	14	96	203
Checkout p95 (unrelated path, same database)	under 400 ms	380 ms	510 ms	870 ms
Drift check	0 rows	0	0	timed out

Message · Monday 11:05 · from the partner's integration manager

"Great news — we rolled your returns flow out to all 1,400 of our sellers over the weekend, the beta feedback was excellent. Let us know if you need anything!"

Commit

Write your five responses. Two of the rows in that table are not about returns at all, and a response that does not act on them within the hour has misread the situation.

Reveal: two coherent responses

Response A — Protect first, then change the driver

The call: act now, then change. Checkout p95 has doubled and replica lag is at 11 seconds: this is an incident affecting a path that has nothing to do with the pilot, and the blast radius the modular monolith accepted has just been drawn for everyone to see. The immediate move is containment, not analysis — the partner cohort is capped back to its previous seller list within the hour using the existing router rule, which is one flag and needs no deploy. Then the design changes: eligibility reads move to the replica behind a staleness-tolerant path, the manual-review queue gains an admission control that refuses to grow beyond its guardrail rather than silently exceeding it, and PF-1 is re-specified and re-tested at the load that now exists.

Artifacts superseded

A5 marked false: "pilot volume is around 40 rps" → observed 412 rps within four hours of a partner decision the team did not know about. The deeper finding is recorded as new A10: "partner-side rollout decisions can change our load by an order of magnitude without notice", which is an organizational assumption, not a technical one.
PF-1's measure changed from 40 rps to 450 rps, with a load test at that figure as the evidence mechanism and an explicit note that the previous number came from a forecast, not a measurement.
New ADR-0014, "Admission control on the returns path", recording the accepted cost: some partner requests are refused with a retry-after rather than queued, which is a visible degradation chosen over an invisible one. Rejected option: scale the database, rejected because it does not bound the manual queue and buys one order of magnitude, once.
OP-1 gains a scenario for the shared database: "if checkout p95 exceeds 400 ms and returns load is above its cap, on-call can shed returns traffic within 5 minutes" — with the cap as a flag, rehearsed.
The extraction decision is formally revisited against its own thresholds. Returns is now well past 15% of the request budget and has caused a checkout impact — two of the five reversal drivers. The revisit is recorded, and the conclusion is to gather one more month of post-cap data before deciding, because a decision made during an incident is a decision made badly.
Rollout contract: partner-originated traffic gets its own stage and its own cap, separate from the customer cohort percentage. It was never a cohort dimension and should have been.
Forecast: pilot completion slips one week, 60% confident, 30 September – 7 October, and the sponsor hears it today rather than on the 30th.

To the sponsor (3 sentences)

A partner switched our returns flow on for all 1,400 of their sellers over the weekend without telling us, which took us from about 40 requests a second to over 400 and began to slow down checkout — so we have capped them back to their original sellers this morning and checkout is recovering. Nothing was lost and no customer was charged incorrectly, but this is the clearest possible signal that we sized the system against a forecast rather than a measurement, and we are re-testing it at ten times the load before we lift that cap. The pilot finish moves out by about a week to 30 September – 7 October, and I would like your help getting the partner team to agree that rollouts of our features get announced to us first.

To engineering (5 bullets)

Cap the partner cohort now, before any analysis. It is one flag, no deploy, and checkout is the priority — returns degrading is a pilot problem, checkout degrading is a company problem.
The drift check timing out is the most dangerous row in that table. It is our only evidence the dual write is intact, and it went blind at exactly the moment the system was under most stress. Get it running against the replica before anything else.
Admission control on the manual queue: refuse past the guardrail with a retry-after, do not queue. A guardrail that silently absorbs 203 cases is not a guardrail, it is a number in a document.
PF-1 is re-specified at 450 rps and re-tested. The 40 came from a forecast and we have now paid for the difference between a forecast and a measurement.
We are formally revisiting the extract-or-not decision, and we are not deciding it this week. Two of its five reversal thresholds have been crossed; that is a reason to gather data, not a reason to redesign during an incident.

Response B — Take the volume; renegotiate the pilot instead

The call: change what the pilot is. The reading here is that 1,400 sellers arriving unannounced is a demand signal worth more than the pilot's tidiness: the initiative exists to find out whether self-service returns resolve faster, and it has just been handed ten times the evidence. Rather than capping back to a comfortable load, cap to a sustainable one — around 150 rps, found by measurement this morning — protect checkout with a hard priority separation, and convert the pilot from a percentage rollout into a capacity-bounded one that admits as much real traffic as the system can serve within its guardrails. The pilot's cohort definition changes; its outcome and guardrails do not.

Artifacts superseded

A5 marked false and A10 added, as in Response A. This part is not optional in either reading.
Frame v0.5 → v0.6: the pilot cohort is redefined from "a percentage of eligible domestic customers" to "as much eligible traffic as can be served within the guardrails, capped by admission control". Non-goals unchanged. This is a real change to what the initiative promised and is versioned as one.
New ADR-0014, "Capacity-bounded pilot with priority separation from checkout", recording the rejected option — cap partners back and rerun the original plan — and why it was rejected: it discards the best outcome evidence the initiative will get, and the partner may not offer it twice.
PF-1 re-specified at 150 rps sustained with a documented degradation mode above it, rather than at a peak the team has not yet proven. Honest, and lower than Response A's 450 because it is a measured number rather than a target.
Rollout contract: percentage stages replaced by capacity stages with the same entry, observation, advance and reversal structure. The manual queue guardrail becomes the binding constraint, because 203 cases in four hours will exceed 50/day at almost any admitted volume — so agent capacity, not database CPU, sets the cap.
Forecast: the pilot outcome now arrives earlier — 90% confident by 20 September, because volume is no longer the limit on statistical confidence — while full rollout moves to Q4 and is now a capacity project rather than a pilot stage. Two dates, stated separately.

To the sponsor (3 sentences)

A partner enabled our returns flow for 1,400 sellers overnight and we briefly saw ten times the traffic we designed for, which began affecting checkout — that is contained now, and the honest reading is that we have just been handed far more real evidence than the pilot was ever going to produce. Rather than turning them off, I want to serve as much of that traffic as we safely can behind a hard cap that protects checkout, which means we will know whether this works by around 20 September instead of the end of the month. The constraint that binds is not our servers, it is our agents: at this volume the manual review queue would run at four times its limit, so the cap is set by how many cases the team can actually review, and I need to know whether you want to fund more of that capacity.

To engineering (5 bullets)

Cap now, but cap to a measured sustainable number, not back to 40. Find it this morning with checkout p95 as the binding signal, and write down how you found it.
Priority separation from checkout is the precondition for everything else here. If returns can still degrade checkout after today, the cap is the only control we have and we should not be arguing for more volume.
The manual queue is the real limit. 203 cases in four hours against a 50-a-day guardrail — admission control is sized by agent capacity, and that is a staffing conversation, not a scaling one.
Rollout stages become capacity stages with the same four parts. Do not lose the reversal criteria in the translation; that is where this kind of change usually leaks.
Two forecasts from now on: the outcome date and the full-rollout date. Conflating them is how "the pilot worked" turns into "so launch it everywhere" without a capacity plan.

Scoring notes · what a strong response makes explicit

That checkout is degrading. Two rows of the table are about a path the initiative does not own, and containment comes before analysis. A response whose first action is a design discussion has let a pilot damage the business.
That the drift check timed out. The safety mechanism for a live migration went blind under load, which means the dual write's integrity is currently unverified. It is the least obvious row and the most consequential.
That the manual review queue — not database CPU — is the guardrail that breaks first, and that its fix is admission control or staffing, neither of which is an architecture change.
The organizational finding, not just the technical one: a partner can change your load by an order of magnitude without telling you, and the fix is a notification agreement plus a per-partner cap, not a bigger database.
That the extract-or-not decision has crossed two of its own stated reversal thresholds, that this is recorded, and that it is not being decided during an incident. Writing thresholds and then ignoring them when they trigger is worse than never writing them.
Whether the new PF-1 number is a target or a measurement. Response A picks 450 as a design target to test against; Response B picks 150 as what it has actually observed sustainably. Both are defensible; asserting an untested number as a driver measure is not.

Simulation 8 · The guardrail

The outcome improves while an important guardrail degrades

Starting position

Week 7, at the 50% stage. The frame's guardrails are: fraud loss no worse than baseline, manual review queue ≤ 50/day, WCAG 2.2 AA on critical flows. The rollout contract's advance criterion for 100% reads "all thresholds hold at each stage". The pilot is due to reach 100% on Thursday and the closeout review is scheduled for 30 September.

The injection

Weekly outcome and guardrail report · week 7

Measure	Baseline	Target	Week 5	Week 6	Week 7
Median resolution, eligible cohort	72 h	under 4 h	3.4 h	3.1 h	2.9 h
Support contact rate	41%	−25% (to 30.8%)	36%	32%	29.7%
Fraud loss per 1,000 returns	EUR 610	no worse	EUR 640	EUR 890	EUR 1,160
— of which: refund-before-receipt path	EUR 95	—	EUR 130	EUR 260	EUR 420
Manual review queue, cases/day	—	≤ 50	34	41	47
Repeat returners (3+ in 90 days) as share of volume	2.1%	—	2.4%	4.9%	7.8%

Note from the fraud analyst: "The growth is concentrated in the automatic refund-before-receipt path for items under EUR 150 — that is 4% of returns and 59% of the new loss. Repeat returners are the clearest signal; a rule on returns in the last 90 days would catch most of it. I would want two weeks to be sure the trend is real and not a seasonal artefact, but I would not wait two weeks before doing something."

Commit

Write your five responses. This is the hardest of the eight, because both defensible answers are uncomfortable and the wrong answer is the comfortable one: advance to 100% on Thursday because the headline numbers are good.

Reveal: two coherent responses

Response A — Hold the stage; fix the mechanism, keep the guardrail

The call: pause exposure, change the mechanism. The rollout contract says all thresholds hold at each stage, and fraud loss is at 1.9× baseline. Thursday's advance does not happen. But the analyst has localized the problem precisely — 4% of returns producing 59% of the new loss, on one path, with a clear signal — so this is not a reason to question the initiative; it is a reason to fix one rule. Refund-before-receipt is restricted to customers with fewer than three returns in 90 days, which the analyst's own numbers say catches most of it; everyone else follows the existing receive-then-refund path, which is slower but is what the baseline system did anyway. The 100% stage moves to the following week, conditional on two weeks of fraud data trending back toward baseline.

Artifacts superseded

Rollout contract: the 100% stage gains an explicit entry condition — "fraud loss per 1,000 returns within 15% of baseline for 10 consecutive days" — replacing the general "all thresholds hold", which was too vague to act on under pressure. This is the artifact the situation proved inadequate.
New ADR-0015, "Restrict refund-before-receipt by return frequency", recording the rejected options: remove refund-before-receipt entirely (rejected — it is a meaningful part of the resolution-time gain), and raise the fraud tolerance (rejected here, and it is exactly what Response B does).
Assumption map: new A11, "automatic refund before receipt does not materially increase fraud", marked false with the week-6 and week-7 numbers.
MO-1 is now load-bearing and gets exercised as intended: this is a one-rule policy change, one engineer, one day. If it takes longer, that is a finding about the module boundary and goes in the closeout.
Outcome telemetry gains the repeat-returner share as a standing measure, promoted from an analyst's ad-hoc query to a dashboard line with a threshold. It was the leading indicator and nobody was watching it.
Forecast: 100% moves from this Thursday, 18 September, to no earlier than 2 October — ten clean days is the new entry condition and the clock starts when the rule ships. The closeout review stays on 30 September and will report the pilot at 50%, with the reason. The outcome targets are unchanged and both are already met.

To the sponsor (3 sentences)

The pilot is working better than we targeted — returns now resolve in under three hours against a 72-hour baseline, and support contacts are already below the number we promised — but fraud losses have nearly doubled and we said we would not let that happen, so we are not going to 100% on Thursday. Our analyst has traced almost all of it to one feature, refunding before we receive the item, being used repeatedly by a small group of customers, and we are restricting that to people who have not made three returns in ninety days, which is a one-day change. Full rollout moves from this Thursday to 2 October at the earliest, and I will report the pilot at 50% at the review on the 30th with these numbers, because I would rather show you a held gate than a good headline.

To engineering (5 bullets)

Thursday's advance is off. The contract said all thresholds hold; one does not. We do not get to interpret that generously because the other numbers are good.
The rule: refund-before-receipt requires fewer than 3 returns in 90 days. One rule, one day, inside the policy module — this is the MO-1 driver being cashed in, and if it takes longer than a day, that is a finding worth writing down.
Promote the repeat-returner share to a dashboard line with a threshold. It moved in week 6, a week before the loss did, and nobody was looking at it.
The 100% entry condition is now specific: fraud within 15% of baseline for 10 consecutive days. Vague gates fail under exactly this pressure, which is what just happened to us.
Do not touch the outcome measures. Both targets are met; the temptation to celebrate them louder because the fraud number is bad is real and it is how guardrails get quietly abandoned.

Response B — Renegotiate the guardrail, in the open

The call: change the guardrail, with a sponsor decision and a hard ceiling. The reading here is that "fraud loss no worse than baseline" was written before anyone knew what the initiative would be worth, and holding it now costs something real: the contact-rate saving is worth roughly EUR 22 per avoided contact across the eligible volume, and the incremental fraud is EUR 550 per 1,000 returns. If the arithmetic favours the trade — and at pilot volumes it does — then refusing to reopen a guardrail because it is written down is bureaucracy, not judgment. But the renegotiation must be explicit, owned by the person who owns the fraud budget, ceilinged, and reversible: the guardrail becomes "fraud loss per 1,000 returns not to exceed EUR 1,250" with a hard stop that turns the path off automatically, plus the frequency rule from Response A because it is a day's work and reduces the loss regardless.

Artifacts superseded

Frame v0.5 → v0.6: the guardrail "fraud loss no worse than baseline" is replaced by "fraud loss per 1,000 returns ≤ EUR 1,250 — above today's EUR 1,160, with headroom stated rather than assumed — and the refund-before-receipt path disabled automatically on breach". The changelog line names the sponsor and the finance owner as the deciders and records the arithmetic. This is the one change in these eight simulations that a lead may not make alone, and the response says so.
New ADR-0015 as in Response A (the frequency rule), plus ADR-0016, "Automatic disable of refund-before-receipt on fraud ceiling breach" — the mechanism that makes the renegotiated guardrail safe rather than merely looser.
Assumption map: A11 marked false, identically. Both responses agree on what the evidence says; they disagree about what to do.
Rollout contract: 100% advances on Thursday, with the new ceiling as an entry condition and the automatic disable rehearsed before the stage opens. Rehearsed, not merely implemented.
Closeout review agenda gains a standing item: revisit the ceiling with three months of data, since a number set on two weeks of pilot evidence is provisional by construction.
Forecast unchanged: 100% on Thursday 18 September, closeout 30 September.

To the sponsor (3 sentences)

Returns now resolve in under three hours and support contacts are already past our target, but fraud losses have risen from EUR 610 to EUR 1,160 per thousand returns — and I need a decision from you rather than making it myself, because "fraud no worse than baseline" is a promise we made to you and I am asking to change it. My reading is that the contact-rate saving is worth more than the additional loss at these volumes, so I would set a hard ceiling of EUR 1,250 per thousand returns with the system switching the risky path off by itself if we hit it, plus a rule limiting it for frequent returners that should bring the number down anyway. If you would rather hold the original guardrail, we hold at 50% and go to full rollout a week later — both are fine with me, but I do not want to advance on Thursday without you having chosen.

To engineering (5 bullets)

We are asking to change a guardrail, not quietly exceeding one. Until the sponsor answers, Thursday is not confirmed — do not prepare the 100% flag flip as if it were.
Build the automatic disable first and rehearse it. A raised ceiling with no enforcement is just a higher number in a document; the disable is what makes the trade safe.
Ship the frequency rule regardless of the decision. It is a day, it reduces loss under either guardrail, and it does not depend on the negotiation.
Promote the repeat-returner share to a dashboard line with a threshold. It led the loss by a week.
The ceiling is provisional on two weeks of pilot data and is on the closeout agenda for revisit with three months. Write that into the frame so the number does not calcify by accident.

Scoring notes · what a strong response makes explicit

That advancing on Thursday is not available without either fixing the guardrail breach or explicitly renegotiating the guardrail. Both reveals refuse the comfortable option; a response that advances because the outcome numbers are good has failed the central criterion of this whole page.
Who owns the decision. Changing an outcome target or a guardrail is a sponsor's call, not a lead's — Response B's defining feature is that it asks rather than decides, and offers the alternative it would accept.
The arithmetic, if the guardrail is renegotiated: what the outcome gain is worth, what the additional loss costs, over what volume, and how uncertain both are. A renegotiation without numbers is a preference.
A ceiling and an automatic enforcement mechanism. A guardrail relaxed without a hard stop has been abandoned rather than adjusted, and the difference is the enforcement, not the number.
The leading indicator that was visible and unwatched. Repeat returners moved in week 6, a week before the loss; promoting it to a monitored line is the systemic fix, and the proximate rule change is not.
That the analyst's caveat is handled rather than ignored: two weeks would confirm the trend, and both responses act now anyway, with a stated reason — the loss is compounding and the frequency rule is cheap and reversible either way.
What goes in the closeout regardless of the choice: an assumption that was false, a guardrail that was either defended or repriced, and a monitoring gap that let a leading indicator run for a week unobserved.

Which rubric dimensions each simulation exercises

Score your committed answers with the capstone rubric. Each simulation loads two or three dimensions hardest; a consistently weak score in one column across several cards is a specific practice recommendation rather than a general verdict. Across the eight, every one of the twelve dimensions is exercised at least once.

Simulation	Primary dimensions	What a lead-ready answer shows here
1 · Vendor latency and availability	Alternatives and trade-offs · Architecture drivers · Data, contracts and failure semantics	Separating a falsified input from a falsified design, and re-specifying a driver measure with its dependent tests and alerts.
2 · Deadline shortened by four weeks	Delivery sequencing · Problem and outcome clarity · Technical communication	Moving the declared variable, holding guardrails under pressure, and stating what a smaller sample can and cannot prove.
3 · New trust boundary	Testability, observability, security, reversibility · Data, contracts and failure semantics · Module boundaries	Updating the model and not only the instance; negative acceptance examples as the deliverable; handling data already collected.
4 · Research disproves an assumption	Problem and outcome clarity · Product, design and engineering collaboration · Delivery sequencing	Weighting evidence by strength, changing the problem statement rather than the target, and pre-committing a decision rule.
5 · Regulator changes retention	Data, contracts and failure semantics · Domain fidelity · Operational ownership	Resolving an apparent conflict by modelling — separating the substantiating record from the person — and sequencing against a migration in flight.
6 · A team cannot deliver its dependency	Delivery sequencing · Product, design and engineering collaboration · Operational ownership	Distinguishing a lost control from a lost detection mechanism, and respecting another team's constraint instead of escalating through it.
7 · Volume ten times the forecast	Architecture drivers · Coupling and cohesion · Testability, observability, security, reversibility	Containment before analysis, noticing that the safety mechanism went blind, and honouring a previously written reversal threshold without deciding inside an incident.
8 · Outcome improves, guardrail degrades	Problem and outcome clarity · Technical communication · Operational ownership	Refusing the comfortable advance, and either defending the guardrail or repricing it in the open with a ceiling, an owner and an enforcement mechanism.

Run one of these on your own initiative

Write your own injection. Take the initiative you are leading now and name the single piece of evidence that would hurt most — the vendor number you have never measured yourself, the assumption nobody has tested, the dependency you are quietly counting on. Write it as the message you would actually receive. Then answer it in the five parts, before it arrives.
Check whether your artifacts can be superseded at all. Try to write the successor to one of your real decisions. If there is no decision record to supersede, no versioned frame to bump, and no forecast with a stated moving variable, then your initiative cannot record a change — and it will change anyway, invisibly. That is the finding.
Find your unwatched leading indicator. In simulation 8 the repeat-returner share moved a full week before the loss did and nobody was looking. For each of your guardrails, name the measure that would move first. If you cannot name one, or it is not on a dashboard with a threshold, you will learn about the breach from the breach.

Related lessons: 0039 (frames and guardrails), 0043 (driver measures), 0046 (options, spikes, superseding ADRs), 0047 (forecast ranges and dependencies), 0049 (rollout gates and reversal), 0050 (continue, change, stop). Reference: glossary — architecture decision record, architecture driver, quality-attribute scenario, rollout gate, assumption map, product outcome.