# Phase 4 — Initiative Lead Lessons 0039–0050. The lab's slice of the initiative work: make the outcome measurable, make the rollout a decision with evidence behind it, and preserve the reasoning. Phase 4 in the course is mostly artifacts — the frame, the story map, the team charter. Those belong in the capstone (`../capstone/index.html`). What belongs *here* is the part of an initiative that is code: telemetry that answers the outcome question, a gate that fails a release, and a decision record for a choice you actually made in this lab. The initiative's stated outcome: **median resolution time from 72 hours to under 4 hours; support-contact rate down 25% from 41%; the manual review queue stays under 50 cases a day.** --- ## 4.1 Give every request a correlation id **Lessons:** 0049 (Testing, Observability and Safe Rollout), 0021 (Error Handling Strategies) **Do this.** Nothing in this system can be traced. There is no log line anywhere; the audit table records an actor but no request; the notification rows cannot be tied back to the request that queued them; and five `except` blocks record failure as an increment of one shared counter. Generate a correlation id at the edge if the caller did not supply one, carry it through the request context you built in exercise 2.6, put it on every log line, every audit row, every notification record and every response. **Observable outcome.** In `fitness/test_fitness_observability.py`: - `test_every_response_carries_a_correlation_id` - `test_the_audit_log_records_a_correlation_id` And the operational test: run the demo, pick one customer, and reconstruct everything that happened to their return from the logs alone. **Hints.** - The id is part of the request context, not a global. If you kept `contextvars` in exercise 2.6, this is where it earns its place. - Accept an inbound `X-Correlation-Id` equivalent from the caller — the support console will want to pass its own. - Structured logs (one dict per line) beat formatted strings the moment you have more than one field to filter on. - The migration in exercise 3.5 is the natural place to add the column. **Forces to weigh.** Correlation ids cost a field in every table, a parameter in every signature, and a small amount of noise in every log line. They buy the ability to answer "what happened to this customer?" in one query instead of an afternoon. The cost is paid at construction; the benefit arrives during an incident, which is exactly why teams skip it. --- ## 4.2 Instrument the outcome, not just the system **Lessons:** 0049 (Testing, Observability and Safe Rollout), 0043 (Architecture Drivers), 0050 (Steering and Closing) **Do this.** `config.COUNTERS` has seven counters. None of them measures the outcome the initiative exists for. Nothing records when a return was submitted versus when it was resolved, so nobody can tell whether 72 hours became 4. Add outcome telemetry: resolution time per return (submitted → refunded or denied), the fraction resolved without an agent touching them, the daily review queue depth against the 50-case guardrail, and the support-contact proxy you can actually measure here. Pair each with a system-health metric (error rate, carrier failure rate, notification duplicate rate) so a "good" outcome number built on a broken system is visible. **Observable outcome.** In `fitness/test_fitness_observability.py`: - `test_the_outcome_metric_is_measured_not_just_counted` And a function you can call after the demo that prints the four outcome numbers, with units, for that run. **Hints.** - Resolution time needs two timestamps and a definition of "resolved". Write the definition down first — a denied return resolves at denial, and a held return does not resolve when the agent opens it. - The demo currently produces returns in four terminal states. Your metric must handle the one stranded in `APPROVED` by the carrier outage; a return that never resolves is the most important number on the dashboard. - Percentiles, not averages. A median of 4 hours with a p95 of 40 is a different product from a median of 4 with a p95 of 6. **Forces to weigh.** Outcome metrics are lagging and noisy — you learn the median a week after the release that moved it. System metrics are immediate and can all look green while the product fails. You need both, and you need to say in advance which one you would roll back on. That is exercise 4.3. --- ## 4.3 Make the rollout gate a test **Lessons:** 0049 (Testing, Observability and Safe Rollout), 0035 (Stability vs Speed of Change) **Do this.** "We will monitor it" is not a gate. Write the pilot's go/no-go conditions as an executable check: given a window of telemetry, does this release pass? Then write the reversal — a flag, a rollback, or a forward fix — and rehearse it. Suggested gate, adapt it: p95 resolution time under 8 hours; automatic resolution rate above 70%; review queue under 50 for three consecutive days; zero duplicate refunds; carrier failure rate under 2%. **Observable outcome.** A test in `tests/` — name it `test_rollout_gate.py` — that takes a telemetry snapshot and asserts each condition, with a failure message naming which condition failed and by how much. Feed it the demo's own run and watch it fail on the duplicate notifications until exercise 3.3 is done. **Hints.** - The gate is a pure function of a metrics snapshot. That makes it testable with fixtures for "passing week", "queue blew out", "duplicate refunds appeared". - Include at least one guardrail that is not about your feature: the review queue cap is a promise to the support team, and breaching it is a rollback reason even if resolution time is excellent. - A gate with no reversal is a wish. Write down what "revert" means here: a feature flag on automatic approval, and what happens to returns already in flight when you flip it. **Forces to weigh.** A strict gate blocks good releases on noisy data; a loose one lets a regression through. The honest calibration is to run the gate against historical windows before you trust it — and to state the blast radius you are willing to accept between two gate evaluations. Staged rollout narrows that; a big-bang release makes the gate a post-mortem. --- ## 4.4 Write the ADR for one lab decision **Lessons:** 0046 (Options, Spikes and Decision Records) **Do this.** Pick one decision you actually made in this lab where you rejected a credible alternative. Good candidates: - **Expand/contract versus a maintenance window** (exercise 3.5) — the coordination cost with finance is the whole story. - **Versioning the API versus coordinating a flag day** (exercise 3.6) — Conway's law is the deciding force. - **Buying the carrier and owning the market rule** (exercise 3.1) — with the break-even arithmetic. - **Events versus a notification port** (exercise 2.12) — you gave up a readable call graph. Write it as an ADR: context, the options you actually considered, the decision, the consequences you accepted, and the signal that would make you revisit. **Observable outcome.** One file — `solutions/` has a worked example at `solutions/phase-4/ADR-0001-expand-contract.md` — with a rejected option that a reasonable person could have chosen, and a named revisit trigger. **Hints.** - An ADR with one option is a diary entry. If you cannot state the alternative fairly, you have not understood your own decision. - Consequences include the bad ones. "Four deploys instead of one, and a dependency on a team we do not control" is the sentence that makes the record useful in six months. - The revisit trigger must be observable. "If it becomes a problem" is not; "if the order system's p99 exceeds 300 ms" is. **Forces to weigh.** ADRs cost twenty minutes each and are only valuable if someone reads them later, which depends on them being findable and short. Writing one per decision produces a graveyard; writing one per *irreversible or expensive-to-reverse* decision produces a record. Lesson 0046's rule — scale review effort to reversibility — applies to the record as much as the decision. --- ## Done when - `test_every_response_carries_a_correlation_id` and `test_the_audit_log_records_a_correlation_id` are green. - `test_the_outcome_metric_is_measured_not_just_counted` is green, and you can print median and p95 resolution time for a demo run. - `tests/test_rollout_gate.py` exists, passes on a good snapshot and fails on a bad one with a message that names the breached condition. - One ADR exists with a fairly-stated rejected option and an observable revisit trigger. Next: `phase-5.md`.