Systems · Field note 01

Reliability at the edge

The edge is where clean architecture diagrams meet weak signals, power cycles, dust, heat, and long periods without an operator.

Start with bounded failure

A dependable device does not need every component to be dependable. It needs failures to remain local, visible, and recoverable. Put explicit time and memory limits around work. Persist only the state required to resume. Treat every external dependency as temporarily unavailable.

Recovery behavior is part of the product, even when nobody sees it during a successful run.

Prefer useful degradation

Define a smaller mode that still produces value: local control without cloud synchronization, coarse estimates without an expensive model, or buffered events without real-time delivery. A clear degraded mode is easier to test than a collection of accidental fallbacks.

Observe the transitions

Steady-state metrics rarely explain field failures. Record transitions: reconnect attempts, clock corrections, thermal throttling, queue saturation, and changes between operating modes. A short event history often explains more than a large stream of averages.