Software Design Reference Capstone · Reviewer notes

Capstone · Reviewer notes

What Separates the Two Packets

Fourteen artifacts, side by side, with the reason each version is strong or weak stated in forces, evidence and consequences — never in preference — and mapped to the rubric dimension it is scored on.

Every annotation on this page answers one question: what decision does this artifact let a reader make, and what happens when the evidence changes? If a criticism cannot be phrased that way, it is taste, and it does not belong in a review.

How to use this page

Each section quotes a short excerpt from the strong packet beside the corresponding excerpt from the weak packet, then explains the difference and links to the rubric dimension it touches. Read the excerpts first and form your own judgment before reading the annotation — otherwise you are memorising verdicts rather than developing the eye that produces them.

Three things this page deliberately does not say: that the weak version is stupid (it is not — it is normal), that the strong version is the only valid design (several of its choices are defensible either way, and the packet says so), or that more documentation is better (the strong packet is longer because the initiative was real, not because length is a virtue).

1 · Initiative frame

Strong

“Move median time from request submitted to decision communicated for the eligible domestic cohort from 72 hours to under 4 hours, and the support-contact rate for that cohort from 41% to 31% or lower … Baseline source: returns queue export joined to the contact log, 1 June – 31 July, n = 4,812 requests; query owned by Priya, re-runnable.”

Plus a version history: v0.2 changed the outcome event from “automatic approval” to “decision communicated”, because the 200-case sample showed 15% of requests are judgement calls.

Weak

“Vision: Deliver a best-in-class, frictionless self-serve returns experience that delights customers and reduces support load.”

“Success criteria: Portal launched and available to all customers. 90%+ of returns approved automatically. Support tickets down. Positive customer feedback.”

“It is version 1.0 and is expected to be stable; changes go through the change-request process.”

Why the strong version is strong

Three properties make it usable. It is falsifiable: at the end of the pilot someone can compute a number and say the initiative failed. “Delights customers” cannot fail. It has a source line: the baseline names the query, the window, the sample size and the owner, so the closing number is computed the same way as the opening one — which is why the closeout can report a 0.6-point miss rather than a plausible-sounding win. It has a version history: the v0.2 change is the most valuable page in the frame, because it records that the team's first outcome statement would have scored correct behaviour (routing a judgement call to a human fast) as failure. Without the history, the current wording looks like it was obvious all along, and the next lead learns nothing.

Consequence for the design: because the outcome is “decision communicated”, the referral state in section 8 is a success path rather than a workaround, and the four-hour agent SLA becomes a thing the team staffs rather than a thing it hides.

Rubric: Problem and outcome clarity, and Technical communication and decision clarity.

Why the weak version is weak

The failure is not the tone; it is that every success criterion is an output the team controls. “Portal launched” is satisfied by launching a portal nobody uses. “90%+ approved automatically” is satisfied by loosening the rules — the criterion actively rewards the fraud risk the guardrail was supposed to prevent, and there is no guardrail here to stop it. “Support tickets down” has no baseline, no cohort and no denominator, so it can be declared true by picking a fortnight.

The consequence arrives later and is expensive: when evidence shows that 15% of cases need a human, this frame gives the team no way to represent that as anything but failure against “90% automatic”. The predictable response is to automate decisions that should not be automated, or to quietly stop reporting the number. And because the frame is “v1.0, expected to be stable”, learning is routed through a change-request process — a mechanism designed to resist change, applied to the document that most needs to absorb it.

Rubric: Problem and outcome clarity.

2 · Assumptions

Strong

“A1 — At least 90% of eligible requests match a codifiable policy rule. Evidence before: weak; one estimate from the policy owner. Test: classify 200 closed agent decisions, 2 analyst-days. Result: Falsified. 63% codifiable, 22% need the photo rule, 15% judgement calls.”

“A6 — fraud does not rise. No pre-test bought — converted into a rollout guardrail with a control cohort and a stop threshold.”

Weak

“Our returns policy can be expressed as rules in code.”

“Automating approvals will not increase fraud, since the same rules apply.”

“The assumptions were validated with stakeholders at kickoff… No further validation is planned; the team will find out during development.”

Why the strong version is strong

The map is not a list of beliefs — it is a spending plan. Each row carries impact-if-false, evidence strength, a proportionate test, a cost, an owner and a result, so the reader can see why A1 got two analyst-days and A7 got two hours of reading. Two rows are the ones to study. A6's chosen test is no test, because fraud responds to the real path existing, not to a prototype; converting it into a rollout guardrail with a stop threshold is the correct answer, and it is written down as a decision rather than an omission. A8 is deliberately untested because it is already well evidenced — buying evidence for a settled belief is a way of looking busy.

Consequence: A1's falsification produced frame v0.2, the referral state, and slice 3. A3's falsification superseded an ADR. An assumption map that changes nothing downstream was not a map; this one is load-bearing.

Rubric: Problem and outcome clarity and Delivery sequencing and uncertainty reduction.

Why the weak version is weak

These are not assumptions; they are assertions in the grammar of assumptions. “Our returns policy can be expressed as rules in code” has no impact rating, no evidence strength, no test and no owner, so nothing about it can be wrong until it is expensive. The fraud line is worse than untested — it contains a hidden argument (“since the same rules apply”) that is false: the same rules applied by a machine at scale, instantly, with no human noticing a pattern, is a different control environment. Stating the reasoning inline lets it pass without review.

“Validated with stakeholders at kickoff” is the specific move to distrust: agreement in a room is not evidence about the world. And “the team will find out during development” is accurate — they will, at the point where finding out costs a sprint rather than two days. The risk register beside it lists only delivery risks (scope creep, holidays, integration effort); not one row is about whether the intervention works. That is the signature of a plan that has confused shipping with succeeding.

Rubric: Problem and outcome clarity and Delivery sequencing and uncertainty reduction.

3 · Workflow and domain

Strong

Current-state timeline with elapsed time per step: “Policy interpreted … +51:05 (queue wait)”. Then: “68 of the 72 baseline hours sit in two queue waits — and neither is work.”

Hotspot: “‘Approved’ means three different things. Support: policy eligibility passed. Warehouse: inspection passed. Finance: money settled.”

Weak

“The current process is well understood by the team — it is essentially the same flow with an agent doing steps 2 and 3 manually — so it has not been documented separately.”

“Domain entities: Return, ReturnItem, Order, Customer, Label, Refund, ReturnStatus.”

Why the strong version is strong

Mapping one real case with elapsed times converts an opinion (“returns are slow”) into a distribution: 68 of 72 hours are queue waiting, not work. That single fact reshapes the whole initiative — it says the win comes from removing waits, not from making steps faster, which is why the target workflow deletes two queues and why raw speed turns out not to be the thing that moves the contact rate.

The “approved means three things” hotspot is the highest-value line in the section. It is evidence, gathered from people who use the words, that three separate models exist. Acting on it produces three owners of three facts, which is what makes invariant I5 (“only Payments may assert settlement”) enforceable. Deciding instead that everyone should agree on one meaning of “approved” would have produced a single status column owned by nobody — the exact defect the weak packet ships.

Rubric: Domain fidelity and invariant protection, and Coupling, cohesion and dependency direction.

Why the weak version is weak

Two moves cause the damage. First, skipping the current state on the grounds that it is understood. It was not: the strong packet's map found a 19-hour evidence round-trip, a transposed order id in a shared spreadsheet, and a class of request (gift orders) with no rule and no owner. None of those is visible from the to-be flow, and all of them arrive later as “edge cases” during implementation — which is where the weak packet's week-3 status quietly records “Support mentioned that a lot of returns are actually gift orders. Noted for later.”

Second, the entity list is derived from nouns in the brief, not from behaviour. ReturnStatus as an entity is the tell: it makes “status” a thing to be set rather than a machine with legal transitions and actors, and it guarantees the shared-status-column design in section 8. The list also has no owners, no change cadences and no invariants, so it cannot suggest a boundary — every later structural decision has to come from somewhere else, which in practice means from the technology stack.

Rubric: Domain fidelity and invariant protection.

4 · Story map versus backlog

Strong

Backbone across (start request → check eligibility → choose resolution → return parcel → track outcome), variation down. “Pilot slice (row 1) … What it tests: assumption A2 and both halves of the outcome, on 1% of eligible demand. Exit evidence: median resolution and contact rate against a matched control, at n ≥ 120.”

Deferred items each carry the reason they wait and a decision date.

Weak

“RET-1 Create returns database schema and models — Backend — 5. RET-2 Create Return API endpoints (CRUD) — Backend — 8 … RET-8 Build returns request screen — Frontend — 8.”

“Total above the line: 91 points. At a velocity of around 20 points per sprint, that is five sprints, which fits the 10-week timeline.”

Why the strong version is strong

The release line is drawn across the journey, so the first thing that reaches production is a complete path for a narrow population. That shape is what makes the outcome measurable early: the pilot slice can produce a median resolution time and a contact rate, and therefore can end the initiative. It also names, per slice, the assumption it tests and the exit evidence — so “done” is a reading, not an opinion.

The deferred column does real work too. Guest orders are 29% of demand, and writing that number beside a decision date converted a recurring argument into a scheduled decision. Deferral with a reason is a decision; deferral by silence is an unmanaged risk that returns every week.

Rubric: Delivery sequencing and uncertainty reduction.

Why the weak version is weak

The backlog is organised by layer — backend, integration, frontend, platform — which means nothing in it is demonstrable to a user until almost all of it is finished. RET-1 through RET-7 can all be complete and correct while no customer has ever returned anything, so the first honest feedback arrives in sprint 4 at the earliest. Every assumption in section 2 stays alive until then, and every integration surprise lands during the sprint that has no slack.

The estimate compounds it: 91 points at 20 per sprint is not a forecast, it is arithmetic wearing a forecast's clothes. It has no range, no confidence, and no statement of which variable can move — so when the carrier work turns out to be six days larger than believed, there is no pre-agreed control surface and the only remaining lever is quality. Note also that the accessibility review is a ticket (RET-15, 3 points, sprint 5). A guardrail expressed as a low-priority ticket in the last sprint is a guardrail that will be traded away under pressure.

Rubric: Delivery sequencing and uncertainty reduction.

5 · Acceptance examples

Strong

“Given the same Idempotency-Key but a different request body, then the response is 409 with code idempotency_key_reuse. Silently returning the first result would hide a client bug; silently processing the second would create two returns.”

“Given delivery exactly 30 days ago at 23:50 store time and submission at 00:10 the following day, then accepted … This example exists because the first implementation used UTC timestamps and refused it.”

Weak

“Given a valid return request, when the customer submits it, then the system checks eligibility and returns a result.”

“Ineligible returns are rejected with an appropriate message. Errors are handled gracefully. The endpoint responds quickly.”

“Edge cases will be raised by QA during testing.”

Why the strong version is strong

Each rule carries a happy path, a boundary, a real failure, a guardrail and the telemetry that observes it in production. The examples are chosen where the team disagreed or where a mistake is expensive: the timezone boundary exists because a real implementation got it wrong, and the idempotency-conflict example states the reasoning for choosing 409 over either silent alternative — so a future engineer who is tempted to “make it friendlier” can see what they would be breaking.

The consequential ones are E3 and E4. E3 pins what a timeout means (unknown, retry with the same reference), which is the semantics that make the whole carrier design safe. E4 states that a settled refund is never automatically reversed, and gives the reason — money movement belongs to Payments — which is why the strong packet has a settlement_disputed state at all. An example that names the reason survives a rewrite; one that names only the expected output does not.

Rubric: Testability, observability, security and reversibility, and Domain fidelity and invariant protection.

Why the weak version is weak

“Given a valid return request … then the system returns a result” is a tautology: it is true of any implementation, including a wrong one. “Appropriate message”, “gracefully” and “quickly” are three unresolved decisions in a document that looks like it has resolved them, and each will be settled silently by whoever writes the code first.

The structural problem is what is absent: no failure semantics, no idempotency, no concurrency, no authorization case, no accessibility case. So the questions that produce production incidents — what does a timeout mean, what happens on a double submit, what if the refund is already settled — are not answered here, are not answered in section 8, and are therefore answered by the first person to hit them at 02:00. Deferring them to “QA during testing” moves the discovery to the most expensive point and to the person with the least context to decide.

Rubric: Testability, observability, security and reversibility.

6 · Architecture drivers versus NFRs

Strong

“D1 — The carrier becomes unavailable for 30 minutes during peak returns traffic. The returns module and label worker must continue to accept and decide eligible requests, and create 99% of the queued labels within 15 minutes of carrier recovery.”

Rejected as a driver: “‘The system must scale.’ … no credible design in the option table responds differently to ‘scale’, so it distinguishes nothing.”

Weak

“Scalability — The system must be scalable to handle future growth. Performance — Pages should load quickly; API responses should be fast. Availability — 99.9% uptime. Maintainability — Code must be clean, well tested, and easy to change.”

“These are the standard NFRs for customer-facing systems in our estate.”

Why the strong version is strong

Each driver names a source, a stimulus, an environment, an affected artifact, a required response and a number — which is what allows it to discriminate between designs. D1 and D2 together make the synchronous carrier call impossible: at a measured p99 of 11.4 seconds, the 800 ms decision budget cannot be met, and a 30-minute outage would refuse every request. That is the whole content of ADR 0003, and it comes from the driver set, not from anyone's preference.

Two further properties matter. The conflicts are written down (D1 versus D2 on the carrier; D3 versus D5 on log detail), with the resolution and its cost, rather than being discovered by an engineer mid-sprint. And the rejected drivers are listed with reasons — the “must scale” entry is the most instructive, because it shows the test: if no design in your option table responds differently to a requirement, it is context, not a driver.

Rubric: Use of measurable architecture drivers.

Why the weak version is weak

Not one of these nine rows can fail a check, and not one of them distinguishes two designs. “Scalable to handle future growth” does not say which operation, at what load, by when — so it cannot argue against any architecture, and in practice it argues for whichever one sounds most scalable, which is how the microservice in section 7 gets justified without evidence.

“99.9% uptime” looks like the exception because it has a number, but the number is attached to nothing: uptime of what, measured how, and — the question that actually matters here — may label creation degrade while request acceptance stays up? The strong packet's D1 answers exactly that and gets a real design constraint out of it. One global uptime figure hides the distinction, so the team can hit 99.9% while refusing every return during a carrier incident.

The consequence is that this section does no work. Section 7's architecture cannot be traced to any row here, section 12's alerts are generic infrastructure thresholds, and when the carrier turns out to be slow in week 3 there is no written requirement it violates — so it is recorded as “the team is investigating” rather than as a falsified premise.

Rubric: Use of measurable architecture drivers.

7 · Architecture views

Strong

Container table with a column headed “Driver that justifies it existing separately”. Returns module: “None — deliberately not a separate deployable. See ADR 0001.” Label worker: “D1: the carrier call must survive a 30-minute outage … A separate process, not a separate service.”

Plus an enforced import table and two dynamic views, one of them the failure path.

Weak

“C4 — Container diagram: Frontend — React 18, TypeScript, Vite. API — FastAPI, Python 3.12. Database — PostgreSQL 16. Cache — Redis. Queue — RabbitMQ + Celery. Runtime — Kubernetes, 3 replicas, HPA.”

“Sequence diagrams for the main flows will be added to the wiki during implementation. The service is deployed as its own microservice so it can be scaled and released independently … in line with our microservices strategy.”

Why the strong version is strong

The container view's most useful column is the one that is mostly empty: which driver justifies this boundary? The returns module has no such driver, so it is a module rather than a service, and the cost of that choice is written down in ADR 0001 (no independent scaling, no independent release of eligibility). The label worker does have one — D1 — so it becomes a separate process sharing the same artifact and the same database: the smallest boundary that satisfies the driver, with no network hop introduced where a transaction will do.

The import table turns the module diagram into something the build can check, which is what makes D4's claim (“one engineer releases a policy rule in a day without touching an adapter”) measurable rather than aspirational — and the closeout measures it at 4 hours. The second dynamic view is the failure path, including the row nobody draws: what happens when the outage lasts four hours and the fallback is the old manual process.

Rubric: Quality and depth of module boundaries, Coupling, cohesion and dependency direction, and Technical communication and decision clarity.

Why the weak version is weak

This is a technology inventory labelled as a container diagram. It names what will be installed and answers none of the questions a container view exists to answer: what owns which data, which component may write which fact, what the interaction protocols are, and which requirement pays for each boundary. Redis appears with no stated purpose; RabbitMQ appears before the decision that would need it; the arrows in the box sketch carry no labels, so the diagram cannot distinguish a synchronous call from an event.

The load-bearing sentence is the last one: the service is a microservice “in line with our microservices strategy.” That is a boundary justified by a policy rather than by a driver, and it is charged rent forever — a network hop between the returns logic and the storefront session, a second deployment, separate observability, version skew, and a consistency boundary that now spans a process. The strong packet's test (“we accept network, versioning, observability and ownership overhead because independent ____ is required by driver ____”) cannot be completed from this document, because section 6 contains no driver that requires independent anything.

And “sequence diagrams will be added during implementation” inverts the purpose: the dynamic view is how you find out that a carrier timeout can lose an accepted request. Drawn after implementation, it documents whatever was built, including the defect.

Rubric: Quality and depth of module boundaries and Coupling, cohesion and dependency direction.

8 · State, data ownership and contracts

Strong

“refund_requested → refunded — actor: Payments only. I5: no other actor may write this state. Recovery if wrong: None. Money has moved; only a new Payments fact can offset it.”

“I4 — Every accepted return reaches a terminal state within 21 days. Reconciliation condition, not an invariant — detection within 24 hours; owner Nils; escalation to Dana after two consecutive days.”

Compatibility: “reason_code is an open enum … state is a closed enum: consumers branch on it.”

Weak

“status varchar — pending / approved / rejected / shipped / received / refunded.”

“PATCH /api/v1/returns/{id} — Update a return (status, agent notes).”

“Clients should be tolerant readers and ignore fields they do not recognise, so adding fields is safe. Retries are handled by Celery's built-in retry mechanism.”

Why the strong version is strong

The transition table answers, for every arrow, who may cause it, what must be true, who owns the resulting fact, and what recovery exists. Two rows carry most of the value. Only Payments may write refunded, which places the invariant “a refund never exceeds the captured amount” where the captured amount lives — the returns module is not trusted to obey the rule, it is structurally unable to break it. And the refund row's recovery is “None”, stated plainly. Admitting that produced the settlement_disputed state and E4's rule against automatic compensation; a table that had written “rollback” there would have produced a system that silently reverses money.

I4 is classified honestly as a reconciliation condition rather than an invariant, with a detection window, an owner and an escalation — because it is temporarily false by design and calling it an invariant would have been a lie that a transaction cannot fix. The compatibility rules distinguish an open enum from a closed one, which is the difference between a new reason code being free and a new state breaking every consumer.

Rubric: Data ownership, contract evolution and failure semantics.

Why the weak version is weak

status varchar plus PATCH /returns/{id} is the whole defect in two lines: any caller with the endpoint can move a return to any state, including refunded. There is no actor per transition, no precondition, no ownership, and — because the same field carries all three meanings of “approved” that the domain map would have found — no way to tell whether “approved” means policy passed, inspection passed, or money moved. The invariant that a refund cannot exceed the captured amount lives nowhere at all.

“Clients should be tolerant readers, so adding fields is safe” is true of optional fields and false of everything else: a new status value will break every consumer that branches on the enum, and a changed meaning of an existing field will pass every schema check while breaking every reader. Because no compatibility policy distinguishes these cases, the safe change and the breaking change are indistinguishable at review time.

“Retries are handled by Celery's built-in retry mechanism” delegates a domain decision to a library. The mechanism retries the task; it has no idea whether the carrier call is idempotent. With no client reference and no uniqueness constraint, the strong packet's measured outcome — two duplicate labels in ten timeout-retries — is the behaviour this design ships, and nothing here would detect it.

Rubric: Data ownership, contract evolution and failure semantics.

9 · Options and decision records

Strong

Three genuinely different mechanisms compared on drivers satisfied, drivers harmed, delivery cost, operational cost, reversibility and largest unknown. “Option C … Rejected as the primary mechanism; retained as the fallback for a carrier outage longer than four hours.”

ADR 0003 records accepted costs, a named dissent with the condition that would prove it right, a revisit trigger, and “Supersedes ADR 0002 … which remains in the log, unedited.”

Weak

“Decision: We will generate labels asynchronously using RabbitMQ and Celery. Rationale: RabbitMQ and Celery are already part of our standard stack, the team knows them well, and this is the established pattern for background work in our platform.”

“Consequences: Labels are not available immediately; the frontend will poll the status endpoint. We will need to monitor the queue.”

Why the strong version is strong

The option table compares mechanisms, not vendors: fail synchronously, persist and drain, or have a human do it. Because the comparison runs against the named drivers, the rejection of option A is a measurement rather than an opinion, and the rejection of option C is a guardrail arithmetic — it breaks the 50/day manual queue somewhere between the 10% and 50% exposure stages. Option C then survives in a narrower role as the outage fallback, which is usually the most useful thing an option table produces.

The ADR does four things the weak one does not. It states accepted costs (a two-step customer flow that assumption A2 says may cost comprehension, a new on-call surface, a pending state now visible in the contract). It records a named dissent with a disproof condition — Sam's argument that this is over-engineering for pilot volume, plus the number that would have proved him right — and the closeout revisits it and reports that it was not upheld. It carries a revisit trigger in measurable terms. And it supersedes rather than edits ADR 0002, so the history of what the team believed remains readable.

Rubric: Quality of alternatives and trade-off reasoning, and Technical communication and decision clarity.

Why the weak version is weak

One option is presented, and the rationale is familiarity: it is in our stack and the team knows it. Familiarity is a real force and belongs in a comparison — but here it is the entire argument, so the record cannot show what was given up. A reader in six months cannot tell whether a synchronous call was considered and rejected, whether a database-backed outbox was compared against a broker, or whether anyone measured the carrier at all.

The decision is also under-specified in the way that matters. “Celery's retry mechanism handles transient failures” does not say whether the carrier call is idempotent, what identifies one label intent, or what happens if the task is delivered twice — so the duplicate-label failure is not decided against, it is simply unaddressed. And the queue is a broker outside the accepting transaction: if the return commits and the enqueue fails, the return is accepted with no label and nothing to notice. That is a consequence, and the Consequences section instead lists a UI polling detail.

There is no dissent, no revisit trigger and no accepted cost. The three things a decision record exists to preserve — what was true, what was given up, and what should reopen it — are all missing, which makes this a note of what happened rather than a record of why.

Rubric: Quality of alternatives and trade-off reasoning.

10 · Delivery plan and dependencies

Strong

“Path walks — one request deploys, persists, crosses the port, and is traceable end to end from a clean build. Exit: repeatable from a clean checkout; a forced failure is visible within one minute. Retires: integration and deployment risk. Dana · week 2.”

Dependency row: “Asha (platform) · sandbox key with 20 rps · needed week 2 · a test call succeeded 26 Aug · fallback: fake adapter behind the real port · escalate Wednesday → platform lead.”

Forecast: “26 September to 3 October, ~70% confidence … Movable: scope.”

Weak

“Sprint 1 Foundations: schema, models, project setup, auth. Sprint 2 Core backend. Sprint 3 Integrations. Sprint 4 Frontend. Sprint 5 Hardening.”

“Platform team: carrier sandbox credentials. Raised in the platform channel; awaiting response.”

“Estimated completion: 5 sprints from 24 August, ending 30 September. The plan is achievable with the current team.”

Why the strong version is strong

The plan is ordered by what each step retires, not by what it builds, and the walking skeleton is deliberately almost empty of behaviour so that the whole real path — build, pipeline, deployable, migration, module seams, outbox, worker, correlation id, telemetry — is exercised in week 2. It paid immediately: the session cookie not reaching the worker, the migration's table lock, and the 3-day log retention that the privacy review had assumed was 30. All three were cheap in week 2 and would have been schedule-defining in week 9.

Every dependency row has an owner, an exact deliverable, readiness evidence, a fallback and an escalation trigger with a recipient — which is what turns “waiting on platform” into something that either resolves or escalates on a known date. The privacy row is instructive for having no fallback: it is marked as a gate, not a risk, so exposure cannot start without it.

The forecast is a range with a confidence, plus an explicit statement of what is fixed and what may move. Naming scope as the control surface in week 1 is what let the week-3 review take a scope trade as a decision rather than as a crisis.

Rubric: Delivery sequencing and uncertainty reduction, and Operational ownership and initiative closeout.

Why the weak version is weak

The sprint sequence is horizontal: everything backend, then everything integration, then everything frontend, then “hardening”. Integration therefore happens once, late, in sprint 3, and usability happens later still — so the two things most likely to be wrong are discovered when there is no time to respond. “Hardening” as a sprint name is the tell: monitoring, load testing, accessibility and documentation have been separated from the work that needs them, which makes them the first things sacrificed when sprint 3 overruns. It did.

“Raised in the platform channel; awaiting response” is not a dependency plan. It names no person, no date, no readiness evidence, no fallback and no escalation, so it can only be resolved by luck or by becoming a crisis. The strong packet's fake-adapter fallback exists precisely so that this dependency cannot block the skeleton.

“The plan is achievable with the current team” is a claim with no range and no movable variable. When reality arrives, no trade has been pre-agreed, so the adjustment happens invisibly, in the only place with no gate: quality.

Rubric: Delivery sequencing and uncertainty reduction.

11 · Team and decision rights

Strong

“Rollout stop / reversal — final call: any on-call engineer, unilaterally. Must seek advice from: nobody — advice happens after. Deliberately asymmetric: advancing needs agreement, stopping does not.”

“Review: every change reviewed, but the tech lead is not a required reviewer — guardrails are published as the import fitness function and the contract tests.”

“Missing expertise, named: nobody on the team has run a fraud guardrail before … Risk holds the stop authority rather than the team.”

Weak

“Decisions: we work as one team and decide together; anything contentious goes to the tech lead or the PM.”

“Code review: all pull requests reviewed by the tech lead before merge.”

“The team is collaborative and everyone is encouraged to contribute to any decision.”

Why the strong version is strong

Every decision in the packet has a named decider and a named advice list, which is what lets the team consult widely and close quickly — the two things that “we decide together” usually trades against each other. The asymmetry on rollout stop is the highest-value row: advancing exposure requires two people to agree, stopping requires one person to act and explain afterwards. A team that needs permission to stop will not stop in time, and the cost of that asymmetry — an occasional unnecessary stop — is far lower than the cost it prevents.

Two more consequences worth noticing. Distributing code review by publishing the guardrails as executable checks is what makes driver D4's one-day policy release physically possible; a single required reviewer would be a queue in front of the very property being promised. And naming the missing expertise, then placing the fraud stop authority with Risk rather than with the team, removes an incentive problem: the people who want the initiative to succeed are not the people who decide whether its guardrail has been breached.

Rubric: Product, design and engineering collaboration.

Why the weak version is weak

“We decide together, and anything contentious goes to the tech lead or the PM” specifies nothing. It does not say who decides scope against experience, who decides whether a guardrail has been breached, or who may stop a release — so under pressure the decision goes to whoever is most senior, most insistent, or in the room. The failure is invisible while things go well and total when they do not: during an incident, nobody knows whether they are allowed to turn the flag off.

“All pull requests reviewed by the tech lead” converts one person into a throughput limit and a single point of knowledge. It also substitutes a person for a mechanism: the boundary rules and contract promises live in one reviewer's head rather than in checks the build can run, so they degrade the moment that person is on holiday — and there is no import rule, no contract test and no fitness function anywhere in this packet to catch it.

Note what is absent: no WIP limit, no swarm rule, no escalation trigger, no named source of truth per fact, and no consultation path for the expertise the team does not have. The section describes ceremonies rather than a decision system, which is why section 13's status report has no decision in it.

Rubric: Product, design and engineering collaboration.

12 · Tests, telemetry and rollout

Strong

“Stage 3 — 1% of eligible demand, with a 10% holdback control … Advance when: median resolution improved against control; fraud not worse. Pause / reverse when: fraud loss above control by more than 1.0 per 1,000 · Risk.”

“Finding, first rehearsal: the worker read the flag once at start-up and kept retrying the 63 intents already in its backlog. The stopwatch said ‘disabled in 3 minutes’; the carrier's dashboard said we were still calling it eleven minutes later.”

“Reversal is therefore ‘stop the intake’, never ‘undo the week’.”

Weak

“Unit tests, minimum 80% coverage (enforced in CI). Load test before launch: 100 requests/second for 10 minutes.”

“Datadog dashboard: request rate, error rate, p95 latency, CPU, memory, queue depth.”

“Enable the flag for 100% of customers on launch day … If there is a serious problem, disable the flag or roll back the deployment.”

Why the strong version is strong

Tests are selected by risk, so each row names the driver or assumption it defends and where it runs — and the two that catch the most are the ones a coverage target would never produce: a recovery drill run before every stage, and two exploratory charters per slice, one of which found the receipt-reversal case that became E4.

Telemetry is paired. Outcome measures (resolution time, contact rate) sit beside health measures (carrier latency, backlog age) and data-integrity measures (oldest accepted return with no terminal state), and every row names an owner and the decision it can change. A measure that cannot change a decision is a dashboard ornament, and none survived here.

The holdback control is what makes the outcome readable at all: “it improved after we launched” is not a finding, and the closeout's exposed-versus-control comparison is only possible because 10% of customers were deliberately left alone. The reversal rehearsal is the strongest single artifact in the packet — it cost 30 minutes and found that the disable flag did not do what everyone believed. And the contract states plainly what reversal cannot undo: labels bought, refunds requested, emails sent.

Rubric: Testability, observability, security and reversibility, and Operational ownership and initiative closeout.

Why the weak version is weak

Coverage is a proxy: 84% coverage of a rules engine tells you the lines ran, not that the rules match what an expert would decide. The strong packet's equivalent — 200 real agent decisions replayed as tests — is the check that can actually fail for a domain reason, and it is what surfaced the missing gift-order rule. The load test at 100 rps is meanwhile testing a number nobody derived; the real peak is 40 rps, and no requirement anywhere says what latency at that load must be, so the test cannot fail either.

The dashboard measures the system and nothing else. Request rate, error rate, latency, CPU and queue depth can all be green while returns take three days, while the fraud rate doubles, or while every mobile customer abandons the flow. Nothing here would show that the initiative failed, which means the initiative can only be judged by whether it shipped.

Going to 100% on launch day removes the one property that makes production learning safe — a limited blast radius and a population to compare against. And “disable the flag or roll back the deployment” treats reversal as a single action, when the flag stops new intake but does not recall a refund, void a label, or unsend an email. Because the reversal was never rehearsed, whether the flag even stops the background worker is unknown; in the strong packet, rehearsing exactly that revealed that it did not.

Rubric: Testability, observability, security and reversibility.

13 · The weekly review

Strong

“Outcome: not yet measurable — no customer is exposed … Reporting ‘on track’ here would be a claim with no source.”

“Changed decision: ADR 0002 is superseded by ADR 0003 … Dissent, recorded: Sam — at pilot volume, option C solves this with a runbook rather than a queue, a backlog metric, a flag and new on-call surface.”

“Evidence 3 · comprehension test at n = 5 — changed nothing, on purpose. Inconclusive … an inconclusive result that goes unreported becomes a silently confirmed assumption.”

Weak

“Status: Green — on track for 30 September. Sprint 2 completed: 22 of 24 points (velocity 22). Carrier integration started — approximately 80% complete. 12 tickets closed.”

“The carrier sandbox is slower than expected. The team is investigating and will optimise if needed.”

“Support mentioned that a lot of returns are actually gift orders. Noted for later.” · “Blockers: none that will affect the date. Asks: none.”

Why the strong version is strong

The review reports evidence and decisions, and it is willing to say that the outcome is not yet measurable rather than manufacture a colour. Three results arrive and produce three different responses: one decision reversed, one re-ordered, one deliberately not made. A review in which every piece of evidence conveniently supports the existing plan is being performed, not run.

Two mechanics are worth copying. Dissent is recorded with a disproof condition — both Sam's (on the worker) and Marco's (on the appeal flow) come with the number that would have proved them right, and both are adjudicated later against that number rather than by seniority. And the inconclusive result is reported as inconclusive: two of five is a strong-sounding fraction of a sample too small to distinguish a wording problem from two participants, and saying so is what keeps A2 alive on the frame's “next evidence” line instead of quietly graduating to a fact.

The response to that weak evidence is also correctly proportioned: change the wording, because it is one line and fully reversible; do not change the architecture, because that is neither. Cheap-and-reversible on weak evidence is fine; expensive-and-irreversible on weak evidence is not.

Rubric: Product, design and engineering collaboration, and Technical communication and decision clarity.

Why the weak version is weak

Every line reports activity: points, tickets closed, velocity, coverage. None of it is evidence about whether the initiative will change anything for a customer, and none of it can turn the status from Green to Amber, because no threshold has been defined for any of it. “80% complete” on the carrier integration is the classic false precision — the remaining 20% is the unknown behaviour, which is why the strong packet reports milestones as demonstrable exits instead.

The two most important sentences are filed as minor risks. “The carrier sandbox is slower than expected” is the same observation that superseded an ADR in the strong packet; here there is no latency requirement for it to violate, so it becomes a performance annoyance to “optimise if needed”. “A lot of returns are actually gift orders” is a falsified policy assumption arriving from a domain expert, recorded as “noted for later” — it will resurface as an escaped defect or as an agent quietly overriding the system.

“Blockers: none that will affect the date. Asks: none.” A review with no ask and no decision is a broadcast. Its cost is not the thirty minutes; it is that the forecast keeps its confidence long after the evidence supporting it has gone, so the options that were available in week 3 have expired by the time anyone admits the position has changed.

Rubric: Technical communication and decision clarity.

14 · Closeout

Strong

“Support-contact rate … 31.6% exposed; 40.2% control. A 23% relative reduction. Missed by 0.6 points. Reported as a miss, not rounded into a win.”

“The weakest number in this packet, named. The fraud guardrail's control cohort contains 694 requests … the control cannot detect a fraud change of any plausible size.”

“Manual warehouse receipt confirmation — Debt, recorded. Removal trigger met on 9 November, not acted on. Interest: about 40 clerk actions/day, indefinitely. Owner: Kim. Reviewed quarterly.”

Weak

“Launched on 29 September, one day ahead of plan. Portal is live for 100% of customers. Most returns are now approved automatically. Support has reported a noticeable drop in returns-related tickets. No P1 incidents in the first two weeks.”

“The feature flag remains in place for safety.”

“The team rolls straight onto phase 2 in October.”

Why the strong version is strong

Every number is reported against its own baseline, with the query that produced it and the same join used to compute the baseline — so the opening and closing figures are comparable, which is the property that makes a 0.6-point miss reportable at all. The miss is stated as a miss. The segments that did not benefit are listed, including the fact that the headline result covers 71% of demand because guest orders were never in scope, with the instruction to quote it with that clause attached.

The strongest move is naming the packet's own weakest evidence: the fraud guardrail's control cohort was too small to detect the change it existed to catch. That is a defect in the measurement design, not in the system, and identifying it converts a comfortable “guardrail held” into a specific, owned improvement for the next initiative. Ownership transfer is by name with readiness evidence — Kim's team ran the outage drill unaided before accepting on-call — and temporary machinery is either deleted with a date or recorded as debt with an interest rate and a trigger. The manual-receipt row is the honest one: the removal trigger fired and the work was not done, and superseding the ADR would have hidden that.

Rubric: Operational ownership and initiative closeout.

Why the weak version is weak

Not one claim here is checkable. “Most returns are approved automatically” has no denominator and no comparison; “a noticeable drop in tickets” is a report of someone's impression with no baseline, no window and no cohort; “no P1 incidents” measures the absence of catastrophe, not the presence of value. Since the whole population was exposed on day one, there is nothing to compare against even in principle — the choice made in section 12 has removed the ability to know whether the initiative worked, permanently.

“One day ahead of plan” is the sentence that reveals what was being optimised. The date was never the goal; it was a constraint on achieving an outcome that this document never measures.

Then the operational tail is simply absent. No named steady-state owner, no on-call handover, no runbook acceptance, no access revocation, no removal of the pilot machinery — “the feature flag remains in place for safety” is how a flag becomes permanent, doubling the live state space of every future change with nobody owning its deletion. The retrospective records feelings and two process actions but no organisational memory: which assumptions failed, what the carrier actually does under retry, what the gift-order finding means. Rolling straight onto phase 2 completes the pattern — the next initiative starts with the same untested assumptions, the same status column, and the same 80%-complete reporting.

Rubric: Operational ownership and initiative closeout.

How to compare your own packet

Do this after you have written your own packet, not before. Comparing against a worked example first produces imitation; comparing after produces judgment, because you will recognise the shape of your own decisions in both columns.

  1. Read your packet as a stranger, in one sitting. Do not fix anything yet. Mark, in the margin, every sentence you would have to explain out loud for a reader to understand it. Those marks are where your artifact is carrying meaning that lives only in your head.
  2. Circle every number with no source and every “must” with no owner. This is the same first pass lesson 0039 asks for on a brief; apply it to your own work. Each circle is either a research task or a claim to delete.
  3. Walk the traceability line. Take your riskiest assumption and follow it: outcome → assumption → slice → driver → boundary or contract → decision → milestone → test or telemetry → rollout gate → continue/change/stop. The first place the line breaks is either work nobody needed or a risk nobody is carrying. Name which.
  4. Find your rejected options. Search your packet for a design you considered and did not take, written down with the reason. If there is none, your decisions are currently indistinguishable from preferences — go back to your most consequential choice and write the two alternatives you dismissed in your head.
  5. Find your accepted costs. For each decision, write the sentence “we accept ____ in order to get ____.” A decision with no cost in it has not been examined.
  6. Find your changed decision. Which artifact records something you believed at the start and stopped believing? If nothing changed across the whole initiative, either you learned nothing or you did not write down what you learned. Both are findings.
  7. Find your dissent. Name a person who disagreed, what they argued, and the evidence that would have proved them right. If nobody disagreed with anything, check whether you asked — “what would make this fail?” is the question that produces it.
  8. Test one guardrail's power. Take a guardrail and ask whether the measure you chose could actually detect a change of the size you care about, at the volume you will have. This is the failure the strong packet names in its own closeout, and it is the most commonly missed item on this list.
  9. Score against the rubric, twice. Score yourself on all twelve dimensions in the rubric, citing the specific line in your packet that justifies each score. Then read the corresponding annotation on this page and re-score. Where the two scores differ, the gap is your next practice — not a grade.
  10. Follow the instructions in the capstone assessment for the self-review before the reveal, then turn the result into portfolio evidence with the portfolio guide.
Self-review checklist · a packet is ready when every line is true

Where to go next Blank versions of every artifact in these packets are in the template pack. Score with the rubric and follow the capstone assessment. When you are ready to make it public, the portfolio guide covers anonymisation and the interview narrative. The method behind each artifact is in the initiative playbook.