Adama Caravan: the storage engine

Mar 2022 – Feb 2026 · age 40–44

What it was

The storage engine under Adama's living documents: a single-node, single-threaded log of ordered appends per document. The bytes lived in a pre-sized memory-mapped heap, the metadata in RAM, and the truth in a checksummed write-ahead log that began every rotation with a full snapshot of itself. A thread slept just under a millisecond, then wrote and fsynced everyone's changes at once, and nothing became visible to readers until the disk said yes. When a document went idle it was archived to the cloud and deleted locally, so every reopen was a restore. Jeff wrote it after benchmarking RocksDB against this workload and finding the general tool about ten times too slow. It ran in production, in solo mode and in the devbox, as a single point of failure its author named in public.

Wins, for the age

• At forty, the human looked at a well-loved database, measured it against the real workload, and wrote a replacement that went "Tick. Tock. Woah!" on the benchmark.
• Made restore the main path rather than the disaster path, concurrently with Litestream (2021), which made the same bet on restores.
• Designed recovery as re-execution: replay re-asks the heap for each region and asserts it gets the same one back, a corruption check built into the replay.
• Built a best-fit heap with constant-time coalescing from three maps, and size-class arenas by composing simple heaps.

What it taught

• Benchmark the workload you have, not the one the world has.
• Backups that are never restored are theatre. Make restore the path you exercise every day.
• Acknowledgement order is everything. Snapshot, recover and delete answered success before the page reached disk, and a test named for the bug asserts it is still there.
• A fatal error needs a stop, not a retry. A failed flush retried the same broken page about a thousand times a second.

Genealogy

Ancestors: Career S3: years inside an object store, and a respect for durability. Lamancha IV: a first disk-backed document store, and Raft left as a TODO. ZHeap: a hand-rolled heap that managed its own free space, twenty years earlier. Descendants: Hearth: one-person persistence on a single node, where backup is copying a directory. Part of: Adama.

Epitaph

It kept its bytes in a mapped file and its truth in a log that began with a picture of itself. Ten times faster than what it replaced, it was good for the two or three years its author predicted.