Software Design Reference Master Designer · System Trade-Offs

Lesson 0032 · Master Designer · Module 2

Consistency vs Availability

When the network fails, a distributed system must choose: answer with possibly-stale data, or wait for the truth. The Master's skill is choosing per operation — not per database.

Reading ≈ 9 minPractice ≈ 25 min

Mission tie-in: the second System Trade-Off — where lesson 0011's invariants meet the real world of partitions and timeouts.

Knowledge: the choice under failure

CAP says: during a partition (network failure between nodes), you choose between consistency (every read sees the last write) and availability (every request gets an answer). The catch — Kleppmann's correction: almost no system is purely one or the other, and CAP is a framing, not a product picker. The real question per operation: what does a stale answer cost, and what does a timeout cost?

The principle behind the principle

The two forces are agreement — every read reflects the last write, so no one acts on a fact that is no longer true — and answering — every request gets a response, so no one is blocked by a machine they cannot see. The unit of decision is the operation, and on a good day the field: one read, one thing the caller will do with it.

The mechanism is an epistemic limit, not an engineering weakness. A node that stops hearing from its peer cannot tell "the peer is down" from "the network is slow" from "the peer is fine and my reply is queued". Facing that ambiguity it has exactly two moves: wait for confirmation it may never get, or answer from what it holds locally. Every consistency protocol ever written is a rule for choosing between those two, and both are correct — they differ only in which failure the business can absorb.

The cost is not only paid during outages, which is what most teams miss. Strong consistency means a round trip to a quorum on the critical path, so the latency floor of a read is the network distance between replicas, on every ordinary day. That is why the rule stops applying in two places. If your data lives on one node in one region, there is no partition to trade around and a consensus layer buys you its costs with none of its benefits. And if a stale answer is illegal rather than merely expensive — a regulated disclosure, a credit decision — this is not a cost comparison at all; the answer is fixed before the discussion starts.

OperationStale read costs…Timeout costs…Lean
Account balanceOverspending, legal exposureFrustration, retryConsistency
Product stock displayA wrong "in stock" hintLost sale, empty pageAvailability
Order statusConfused customerRetry-able annoyanceAvailability
Fraud decisionChargebackBlocked purchaseConsistency

The engineering that makes this a design instead of a gamble:

Both of the middle two are code you can point at. The idempotency key buys availability without giving up write safety:

def charge(self, amount: Money, token: str, key: IdempotencyKey) -> ChargeId:
    existing = self._charges.find_by_key(key)
    if existing is not None:
        return existing.id            # the retry returns the first answer
    charge_id = self._gateway.charge(amount, token, key)
    self._charges.record(key, charge_id)
    return charge_id

The detail that makes this work: the key is generated by the caller and reused across every retry of the same intent. A client that mints a fresh key per attempt gets two charges, and the mechanism has bought nothing — the safety lives in the caller's discipline as much as in the server's table.

And staleness, made visible instead of silent:

@dataclass(frozen=True)
class StockHint:
    quantity: int
    as_of: datetime              # the UI can say "as of 2 minutes ago"

A number with a timestamp is an honest answer. The same number without one is a promise the system cannot keep — and the customer discovers that at checkout instead of on the product page.

Worked: the same call, two different answers

A returns service checks eligibility by reading the order system, which is authoritative for orders. The check needs two facts: when was this ordered (the 30-day window) and has it already been refunded. One API call, so most teams treat it as one consistency decision. The order system degrades for 40 minutes. What should the returns service do?

The first thing a lead looks at is not the endpoint — it is each field's mutability and the action it authorises. The order date is immutable: once written it never changes, so a copy from an hour ago is as true as a copy from a millisecond ago. The refund status is mutable and it authorises money leaving the company: a stale "not yet refunded" is the precise input that produces a second refund. Same call, same partition, opposite answers.

def check_eligibility(order_id: str, orders: OrderReader) -> Eligibility:
    snapshot = orders.local_snapshot(order_id)         # date: immutable, cache freely
    if not snapshot.within_days(30):
        return Eligibility.rejected("outside the window")
    try:
        status = orders.refund_status(order_id, max_staleness=timedelta(0))
    except Unavailable:
        return Eligibility.deferred("decision within 4 hours")   # the third answer
    return Eligibility.approved() if status.none_yet else Eligibility.rejected("already refunded")

The alternative a lead rejects is the one that sounds like availability: cache the whole order response for five minutes so the flow keeps working. Price it. At 3,000 returns a day, a 40-minute degradation covers about 83 requests; roughly 4% are re-filings of a request the customer already made, so about three of them can be approved twice. At an average refund of €62 that is €186 — trivial money, and entirely the wrong number to look at. Each duplicate is a manual clawback, twenty minutes of an agent's day, an awkward conversation, and a measurable move in the fraud-loss guardrail the initiative promised not to worsen.

The opposite alternative is rejected too: block every check until the order system returns. Those 83 customers see a spinner and then an error. If only half of them contact support — conservative, since the baseline contact rate for returns is 41% — that is 34 contacts at roughly six minutes each, 3.4 agent-hours bought in exchange for a strictness the date field never needed.

Which is why the honest design has a third answer. "Stale or timeout" is a false binary produced by assuming every request must resolve now. A deferred case — "received, decision within four hours" — keeps the promise the outcome actually made, needs no stale read, and blocks no one. It comes with its own arithmetic: 83 deferred cases in one outage would swamp a manual review queue capped at 50 a day, so deferrals must route to an automatic retry once the dependency recovers, with the agent queue as the exception path, not the default.

Over a year the difference is which conversation the team keeps having. With per-field decisions written down, a new field arrives and someone asks "immutable or authorising?" — a two-minute conversation. With one blanket TTL on the whole response, every new field silently inherits a staleness budget nobody chose for it, and the next duplicate-refund incident is discovered by finance, not by monitoring.

† The framer's trap: saying "our database is CP" and moving on. Products have configurable knobs (sync vs async replication, quorum sizes); the per-operation choice is yours, and most operations in most systems can afford availability with an idempotent write path.
What people get wrong after reading this
"CAP means we pick C or A for our database."
CAP describes behaviour during a partition, which is a rare minority of your uptime. The rest of the time the live trade is latency versus consistency — the "else" half of PACELC — and that one you pay on every single read, forever.
"Eventual consistency means the data is wrong for a while."
It means the replicas converge with no promised bound unless you specify one. The design work is the bound and the disclosure: "at most 60 seconds behind, and the screen says so". Undisclosed staleness is not eventual consistency; it is a bug with a fashionable name.
"Adding a cache is a performance decision."
A cache is a consistency decision wearing a performance costume. Choosing a five-minute TTL is choosing a five-minute staleness budget for every field behind it — including the fields that authorise money.
"Strong consistency is the safe default."
Refusing to answer is also a failure, just one that shows up in a different team's dashboard. A checkout that waits for a quorum it cannot reach has not protected the customer; it has moved the damage from finance to revenue.

When this is the wrong move

Per-operation reasoning is overhead you should decline in three cases. A single-node, single-region system has no partition to reason about: the ceremony buys nothing, and the cache you add "for availability" only adds a way to be wrong. A legally constrained read — a regulated balance, a consent record, a credit decision — is not a cost comparison; the stale answer is not cheaper, it is prohibited. And a user reading back their own action is a special case worth knowing: a technically defensible stale read of a change they just made reads as a broken product. Session (read-your-writes) consistency for that one caller is far cheaper than making the whole operation strongly consistent for everyone.

In the lab: the companion Phase 3 exercise gives you a returns service that caches an authoritative read wholesale and auto-approves from it. You separate the immutable fields from the authorising ones, add the deferred third answer, and prove with a test that a duplicate submission during a simulated outage cannot produce a second approval.

Skill: which way does this operation lean?

During a partition, an eligibility check needs the order date and the refund status. Choose:

A balance read leans towards consistency because a stale answer:

An idempotency key on charges buys safety by:

Practice on your own code

List your system's read operations. For each, write the two cost sentences: "a stale read costs…" and "a timeout costs…". Mark the lean. Then check: where staleness is served, is it visible to the user?

Reveal: an operation ledger

A booking system: seat availability — stale costs a bad hint, timeout costs the sale → cache with "live availability" refresh. Booking confirmation — stale costs double-booking → strict, with an idempotency key per booking attempt. Price display — stale costs a wrong quote → cache with a visible "as of" timestamp. Three reads, three leans, all written down.

Teach it back
  1. In two sentences, explain to a junior why "we chose a CP database" is not an answer to "can this read be stale?"
  2. Find one cached or replicated read in your system that authorises an action — a refund, a discount, an entitlement. Name the staleness budget somebody implicitly chose for it, and what a wrong answer costs once.
  3. Predict the one-year state if it is left alone: which team discovers the first bad decision, and how many of them will have happened by then?

Your win

You can write the two-cost analysis for any read operation, lean per operation instead of per database, and use idempotency keys to buy availability without losing write safety.

Read and watch deeper

Bring your read-operation ledger to your agent-teacher and cost the failures together.