Real network partitions

New executable experiment: real peer connection partitions

First edition A paused processes. This extension keeps all three etcd processes running and puts an owned loopback TCP proxy in front of each peer listener. A partition closes existing cross-group connections and rejects reconnects. Direct client endpoints remain reachable. The proxy reads etcd 3.6.6's first HTTP/1 headers to identify X-Server-From; it does not interpret or simulate Raft messages.

The effect is a controlled connection partition, not arbitrary packet loss, a TLS proxy, or a production network chaos matrix. It modifies no firewall, interface or routing table. Up to 64 concurrent connections per gate and bounded buffers keep the learning runtime small.

Predict before running

1. Write a baseline value and observe it on all replicas.
2. Isolate the current leader from the other two voters in both directions.
3. Ask the majority component to write a new value.
4. Read the isolated member using local serializable mode, then default linearizable mode. Are the results the same?
5. Heal the cut and compare all replicas.
6. As a control, separate all three voters into singletons while keeping every process alive.

Expected contrast: the two-voter component can make progress; the isolated local read may be stale; a completed default read requires the stronger protocol. Three live processes do not by themselves imply a reachable quorum.

Run the original exercise

python3 scripts/install_etcd.py
python3 scripts/run_network.py --repeats 3 --out build/my-network.json
python3 scripts/learning_exercises.py recovery
python3 -m unittest discover -s tests -v

The recovery exercise reads the committed baseline report; your new run stays in build/my-network.json. Change the partition groups in a private working copy to test another schedule. The baseline runner asserts values and caller outcomes independently of its proxy counters. A nonzero cut count alone would not prove consensus behavior.

Observed result and its limit

The saved three repetitions retain: all processes alive, majority write success, isolated local value before-cut, isolated default read and write outcomes unknown, healed replicas all at majority-committed, and no successful write inside the three-singleton deadline. Timed-out writes are reconciled after healing and are not asserted to be rolled back.

Choose D · Peer partitions in the real-run replay · Complete network evidence JSON · Hash manifest.

This is one host, one pinned implementation and a deliberately small fault schedule. It does not prove all partition histories correct, exercise disk loss or restore a backup. The original process-failure, retry and fencing baselines remain separate and unchanged.

Implementation and source references

• lab/peer_proxy.py: owned gates, source classification, group policy and cleanup.
• scripts/run_network.py: partition schedule, independent assertions and selected report fields.
• Pinned etcd stream transport uses the source and destination headers.
• etcd failure modes and MIT Lecture 6 explain the protocol context.