Lesson 0021 · System Designer · Module 2

Practical, Maintainable Error Handling

Lesson 0014 said fail fast at the right boundary. This lesson builds the full strategy: where errors are raised, how they travel, and where they are translated, retried, and logged.

Reading ≈ 8 minPractice ≈ 45 min

Mission tie-in: "design practical and maintainable error handling strategies" — the difference between exceptions as a mechanism and exceptions as a language.

Knowledge: a language, not a mechanism

An error strategy has four layers, each with a job:

1. Raise — in the domain, at the moment of refusal (lessons 0011, 0014).
2. Translate — at boundaries, wrap foreign errors in your vocabulary.
3. Retry — at integration boundaries, where failures are transient.
4. Log and report — at the edges, once, with the error's story intact.

The language is your exception hierarchy — flat, named, and domain-shaped:

class CheckoutError(Exception):
    """Base for everything checkout can fail with."""

class InsufficientStock(CheckoutError):
    pass

class PaymentDeclined(CheckoutError):
    def __init__(self, provider_message: str) -> None:
        self.provider_message = provider_message
        super().__init__(f"payment declined: {provider_message}")

class PaymentGatewayUnavailable(CheckoutError):   # retryable, not a decline
    pass

Each layer then handles exactly what it is placed to handle. Retry belongs to the adapter, because only the adapter knows the failure was a transport problem:

# infrastructure/ — retry wraps the port, never the rule
class RetryingPaymentGateway:
    def __init__(self, inner: PaymentGateway, attempts: int = 3) -> None:
        self._inner = inner
        self._attempts = attempts

    def charge(self, amount: Money, token: str) -> str:
        for attempt in range(self._attempts):
            try:
                return self._inner.charge(amount, token)
            except PaymentGatewayUnavailable:
                if attempt == self._attempts - 1:
                    raise                         # out of attempts: let it travel
                time.sleep(2 ** attempt)          # backoff lives here, not in the domain

Translation and logging belong at the edge, where the error leaves the system and someone outside has to be told something useful:

# api/ — translate once, log once
def checkout_endpoint(cart: Cart, service: CheckoutService) -> Response:
    try:
        service.checkout(cart)
    except InsufficientStock as e:
        return Response(409, str(e))              # a client mistake
    except PaymentDeclined as e:
        return Response(402, e.provider_message)  # the bank said no
    except CheckoutError:
        logger.exception("checkout failed")       # logged once, with the traceback
        return Response(500, "checkout unavailable")
    return Response(201, "created")

The domain in the middle contains no try at all. It raises its own vocabulary and gets on with the rules — which is why its tests are three lines long.

Four rules keep it maintainable: never catch Exception bare; never convert an error to None at the raise site; catch only what you can handle, at the boundary where handling means something; log once, where the error leaves the system.

Worked example: the catch that cost a quarter's reconciliation

A four-person payments team runs a nightly reconciliation over roughly 12,000 settlement rows. Inside the row loop sits except Exception: continue. The job is green every morning.

A senior designer reads the line after the catch first. continue is a statement of policy: this failure has no consequence. So the question is not "should we catch here?" but "who is entitled to decide a settlement row doesn't matter?" Nobody in that loop has the standing, because the loop cannot see the reject rate — and the rate is the only thing separating one bad row from a broken feed. Logging inside the loop is rejected for the same reason: it turns a silent failure into an unread one and leaves the decision where it was.

Price the drift. A 0.3% failure rate is 36 rows a night, about 1,080 unreconciled settlements over a month. At a median €48 a row that is roughly €52,000 of cash the ledger cannot attribute — found by finance at quarter close, not by engineering, and reconstructed through a two-week manual audit. The defect was one line; the bill was a fortnight of someone else's quarter.

The fix moves the decision up one layer and makes the rate visible — landing on the boundary the principle box named, where an accumulated result beats an exception per row.

# application/ — the batch owns the run, so the batch decides
@dataclass(frozen=True)
class RowRejected:
    row_id: str
    reason: str

def reconcile(rows: list[dict], ledger: Ledger) -> BatchResult:
    posted, rejected = 0, []
    for row in rows:
        try:
            ledger.post(parse_settlement(row))      # raises SettlementInvalid
            posted += 1
        except SettlementInvalid as invalid:        # one named failure, handled
            rejected.append(RowRejected(row["id"], str(invalid)))
    if len(rejected) > len(rows) * 0.01:            # a broken feed, not a bad row
        raise ReconciliationAborted(posted, rejected)
    return BatchResult(posted=posted, rejected=tuple(rejected))

Three things changed and only one is code. The catch is specific, so an unexpected failure still travels. The rejects are data, so the run reports "11,964 posted, 36 rejected" to a dashboard rather than to a log nobody opens. And the threshold makes the policy reviewable. Eighteen months later, when a partner silently changed its date format, the job failed on the first night instead of the ninetieth.

When this is the wrong move

Skip the machinery when the layers collapse into each other. A one-file script has no boundary worth translating across. A library raises its own types and stops. Loops where failure is the expected case want an accumulated result, not an exception per item. And don't build a hierarchy before two callers actually distinguish two failures — a base class with one subclass is a naming ceremony, and the second class costs nothing to add on the day the second decision appears.

Skill: where does each error go?

Practice on your own code

Audit one module's exception handling: list every try/except, what it catches, and what it does. Mark each as raise, translate, retry, or log-and-report. Delete the bare excepts; move retries to boundaries; name the hierarchy if there isn't one.

In the lab. The phase-2 checkpoint hands you the returns-resolution service with a bare except Exception around the carrier call that quietly returns None. You classify every catch as raise, translate, retry or report, and move the retry into the adapter that can prove the call is safe to repeat.

Read and watch deeper

• Clean Code, Martin — ch. 7 "Error Handling": the exception-hierarchy and boundary-translation rules.
• A Philosophy of Software Design, Ousterhout — ch. 10 "Define Errors Out of Existence": design so many errors never arise.
• "The Error Model", Joe Duffy — the long primary account of bugs vs recoverable errors, and why that distinction decides where you catch.
• Robust Python, Viafore — Python-specific exception design: when to raise, subclass, and wrap.
• Watch: ArjanCodes YouTube — search "error handling python".
• Next: lesson 0022 — organizing modules and folders.
• Reference: Patterns — Decorator, the shape the retrying gateway above is; glossary — fail-fast.

Bring your module's try/except inventory to your agent-teacher and classify each catch together.