A timeout is an observation made by one participant. It is not a verdict about the rest of the system.
State a system model, separate safety from liveness, and describe exactly what a failure experiment can establish.
Prerequisites: processes, files, and request/response programming. No prior consensus knowledge is required.
Imagine three servers storing a single configuration value. A client sends write(1), then waits. Three executions look identical to the waiting client: the request has not arrived; the server has committed it but its response is delayed; the server crashed before committing. A client deadline cannot distinguish these executions.
client server
send ───── delay ────► not received yet
send ────────────────► commit ─── response delayed
send ────────────────► crash before commit
deadline: same client observationOur baseline model uses authenticated-in-principle, nonmalicious participants, crash/recovery with durable local storage, and messages that can be delayed. The actual teaching runtime uses unencrypted loopback transport and synthetic values. It is not a deployment template. A process pause models a participant that stops taking steps; it does not reproduce every network-partition behavior.
Safety: something forbidden never happens. For example, two completed writes must fit the object's declared order, and a committed configuration must survive a permitted leader crash. Safety must survive arbitrary delays under the algorithm's fault assumptions.
Liveness: something desirable eventually happens. A request eventually completes when enough nodes can communicate, processes are scheduled, and storage makes progress. A finite experiment can show one completion or one deadline miss. It cannot prove an eventuality for all schedules.
In a fully asynchronous model there is no known upper bound on message or processing delay. A slow node is indistinguishable from a crashed one. The FLP result constrains deterministic consensus termination in that model with a possible crash. It does not say useful consensus implementations cannot exist. Practical protocols rely on favorable timing periods for progress while preserving their safety rules during bad periods.
Before running code, list: which processes can fail, whether their disks survive, which endpoints the client uses, the deadline, and the invariant. Use the real quorum experiment: one etcd leader is killed, its data directory remains, and two other voters continue. Predict whether an acknowledged write can disappear and whether an in-flight write has a known result.
fault: one owned process SIGKILL
survives: its WAL directory, two voters
check: old acknowledged value; new write; restarted member catch-up
not checked: disk destruction, Byzantine messages, multi-host outageInterpretation: a successful new write establishes progress in this run after failover. A deadline miss establishes only that this caller did not get success in its observation window.
No. Without a delay bound, the server can always be slower than the chosen timeout. A timeout can trigger a recovery policy; it cannot establish that the old participant has stopped acting.
Crash recovery with an intact WAL and permanent loss of every replica holding a record have different guarantees. “Restart succeeded” cannot substitute for a backup-restore test.