Adama net: the cluster protocol, routing, and gossip

Jan 2022 – Apr 2026 · age 40–44

What it was

The network layer that let Adama's machines talk to each other. Each pair of hosts shared one mutual-TLS socket, and every frame carried a control byte and a stream id, so many requests and long-lived document streams multiplexed over it. Seventy-one message types were compiled from annotated Java classes into generated readers and switch dispatch. Gossip, the failure detector that tracks which machines are alive, rode on the same socket as just another stream. Backends advertised which spaces they could run, rendezvous hashing picked a host for each document, and a "findbind" let the chosen host claim any archived or unknown document with one conditional SQL UPDATE. It replaced gRPC in March 2022, ran in production for years, and cut warm connects from 462 to 12 milliseconds.

Wins, for the age

• At forty, the human designed a failure detector, routing and a wire format from first principles, and published the measurements on a public blog as they went.
• Hash-chain gossip: membership snapshots hashed so peers that agree settle in three small messages. Serf and memberlist (2013) gossiped first, but this quick-and-slow split is uncommon.
• Found on their own the rendezvous hashing GitHub's load balancer (2016) had used, then turned a cross-region lookup into a good guess plus a compare-and-set.
• Swapped out a framework only after measuring: fifteen to forty-five percent better at p95, and half the proxy CPU.

What it taught

• Replace the framework when you can measure the win, and keep measuring after.
• Failure detection wants two speeds: quick when nothing changed, slow only when membership did.
• The fastest lookup is a good guess plus a conditional write.
• Identity is only as strong as its scheme. One shared fleet certificate meant every sibling was trusted, and the second region stayed five comments.

Genealogy

Ancestors: Async Serialization: length-prefixed frames pushed down a socket, the first protocol. Career Facebook: streaming fabrics at scale. Goat: a game master pushing JSON to every screen over tiny frames. Z Multicast Toolbox: peers discovering each other without a server. ZNetFS: many peers, many connections, chunked transfers by hash. Descendants: none on record. The protocol, gossip and findbind wait here for whoever wants them. Part of: Adama.

Epitaph

Seventy-one messages on one socket per pair of machines, gossip riding along as just another stream, and a host that said "mine" to any document nobody else had. A warm connect took twelve milliseconds.