Heap Snapshot Prototype

Point-in-time investigation of snapshotting a paused run's heap at an idle point and restoring it in a fresh process

The decision to build snapshots, and how, is ADR 0148. This page records the prototype and its measurements as of 2026-10-18, including the recommendation made then.

Executive Summary

• A structured object-graph snapshot taken at an idle pause point (no frame running, the run waiting on a host promise) restores correctly in a fresh process for all 15 measured workloads: closures, async functions between awaits, a suspended generator inside for...of, a module suspended by top-level await, classes with private fields, Promise.all, RegExps, typed arrays, Maps and Sets, and five AWFY programs.
• Restoring costs about 2-20 ms for small and medium heaps, most of it creating the engine and compiling the module, which a replay pays too. The graph itself restores at roughly 1.3-1.8 µs per serialized object on the larger heaps.
• A snapshot beats replaying from the start only when the run did much more work before the pause than it keeps alive: 6-650x for long computations with small state (SSR, markdown, image processing, AWFY Richards, Havlak, Json). Short I/O-bound runs that pause within a few milliseconds replay as fast as they restore (0.5-1.0x), and heap-heavy data processing gains only about 3x.
• The render and data heaps (SSR, markdown, the JSON and image workloads, the agent loop) write 0.3-0.7 bytes per byte of live heap, and zstd at level 3 brings them to 3.5-6.5% of that. Four small, object-heavy AWFY heaps write 1.3-1.6 bytes per live byte and compress to 22-26%.
• At the test suite's sampled await points, every heap held classes the prototype has no codec for (module namespaces, error objects, async generators, async context, test-runner natives). The prototype covers the core language; the long tail is the built-in library.
• The prototype took about 1.2 agent hours of active work; production quality was estimated at about 30 agent hours (range 20-45) plus a standing cost on every future built-in. The recommendation on 2026-10-18 was to keep replay as the guarantee and wait for real resume times before building; ADR 0148 records the decision taken since.

Question

The use case is an embedder that pauses a run while it waits for a host answer (a tool result, a human approval) and resumes it later, possibly on another machine. Replaying the program from its start, answering earlier host calls from a log, is the guarantee. The question is whether a heap snapshot taken at the pause can be a faster way to resume, what it costs, and how much work a production version is.

The design under test is a structured graph serialization at idle pause points, with replay as the fallback whenever the snapshot refuses:

• Snapshots are taken only at idle points: no bytecode frame is running and the run waits on a host promise, so the only live control state is in heap objects (suspended async frames, generator continuations, promise reactions). The design also requires an empty microtask queue. The prototype takes its snapshot in an extension's WaitForIdle, which the engine calls before it drains the queue, and does not check that the queue is empty; the measurements below do not establish that condition.
• Each object is written by a codec its class registers. A class without a codec, or a state a codec does not handle, refuses the snapshot.
• Built-ins are not written. They are named by a path from a root of a fresh engine and resolved by name on restore.
• Restore recompiles the same source, checks the module's bytecode hash, skips the module body, rebuilds the graph and settles the pending host promises.

What Was Built

The prototype lives on the prototype/heap-snapshot branch.

The format is a header (magic, format version, SHA-256 of the entry module's bytecode), the names of the built-ins the snapshot refers to, a table of objects (tag, payload) and the roots: the entry module's lexical bindings and the pending host-call promises. Primitives are written inline. Restore runs three passes over the object table: allocate an empty shell per object, fill each payload (references resolve to shells), and finish state that needs other objects filled first. Every shell is rooted until the restore ends.

Codecs exist for: plain and literal objects (descriptors, accessors, integrity levels, symbol keys), arrays with holes, Map, Set, compiled classes and their instances (private names are remapped to the restored class's brand), bytecode functions with their upvalue cells, generator and async continuations (registers, local cells, handler stack, resume state), await handlers, promises with their reactions and capabilities, Promise.all's state and element functions, native functions whose callback is registered by name (promise resolving functions, Promise.all), ArrayBuffer (resizable, detached, immutable), typed arrays, array iterators, RegExps (compiled again from source and flags), the error stack slot of a plain object, local and registered symbols, and a module body suspended by top-level await.

Built-ins are numbered by a breadth-first walk from the global object, the global scope, the realm slots, the realm's shared prototypes, every live builtin's own object and the well-known symbols. Paths separate their segments with a control character, because realm slot names contain dots. The walk does not materialize lazy members: materializing every lazy global (Intl, Temporal, the bootstrap shims) cost about 30 ms per engine in a production build, against about 1 ms for the walk. Before a snapshot the table is walked again, so built-ins the run materialized since are named too; on restore, a name the table does not hold is resolved by walking its path, which materializes exactly the members on it. Realm slots filled on first use (an array iterator's prototype) are created through registered materializers. A fingerprint of every built-in taken at engine start refuses a snapshot when the script changed a built-in.

Two checks reduce the risk of a wrong snapshot, without removing it. The accounting check collects, then refuses when any live object the run created is neither serialized nor a built-in; this caught native-held state the walk missed, but it detects objects a codec missed, not fields. The reader bounds-checks every count, index and payload length and refuses rather than guessing, which protects against malformed input, not against an incomplete codec. The 15 output comparisons below are point-in-time coverage of those workloads, not a general guarantee.

Problems Found on the Way

Method

Machine: Linux x86_64, 32 cores, FPC 3.2.2, production build (-O4) of the prototype/heap-snapshot branch at f02d802b (the probe and the harness), measured on 2026-10-18. The AWFY programs came from a local are-we-fast-yet checkout whose revision was not recorded, and the raw samples were not kept: treat the figures as an informal point-in-time measurement. Each mode ran 5 times; the table shows medians. To re-run:

./build.pas --prod heapsnapshotprobe
cp build/GocciaHeapSnapshotProbe /tmp/probe   # keep the binary fixed while building other targets
bun scripts/heap-snapshot-measure.ts workloads /tmp/probe --reps=5 \
  --awfy-dir=<are-we-fast-yet>/benchmarks/JavaScript --json=results.json

For each workload the harness runs the program uninterrupted in run mode (the replay), takes a snapshot at the chosen pause in snapshot mode, compresses the file with gzip level 6 and zstd level 3, restores it in a fresh process in restore mode, and checks that the output printed before the pause plus the output after the restore equals the uninterrupted output.

• Live objects and live heap: every collector-managed object the run created that survives a collection at the pause, and the collector's byte count for them (instance sizes plus charged external bytes).
• Snapshot objects: the objects in the snapshot's table. Strings, numbers and booleans are written inline and not counted.
• Serialize: walking the graph and writing it, including the re-walk of the built-in table. The accounting check is reported separately below.
• Restore: rebuilding the graph from the file. Restore total: from the start of the process's work to the moment the run can resume, including engine creation, the built-in table and compiling the module.
• Replay: from the start of the process's work to the same pause in an uninterrupted run. It includes engine creation and compilation, as a replay would, and answers earlier host calls immediately, as a replay from a log would.

The AWFY workloads use the repository's AWFY bundle builder and the AWFY lane's compatibility flags; each runs a number of benchmark iterations, pauses on a host call, then runs one more verified iteration. json-data-x4 is json-data with four times the orders.

Results

The workloads, in fixtures/heap-snapshot/:

Other measurements, medians unless stated:

• Built-in table: 1.0-1.4 ms per engine, both at snapshot and at restore, down from 28-37 ms when the walk materialized every lazy member.
• Accounting check: under 1 ms for most workloads, 3.9 ms for markdown, 31 ms for json-data and 118 ms for json-data-x4 (a full collection plus a scan of every managed object).
• zstd level 3 compresses in 0.04-21 ms, one measurement per workload; the biggest snapshot (29 MiB) took 21 ms.
• Engine creation, the built-in table and compiling take 2-5 ms for the in-repository workloads and 8-19 ms for the larger AWFY bundles. A replay pays this too.

Reading the Results

Restore time follows the number of serialized objects: about 1.3-1.8 µs per object on the larger heaps (more where objects hold many strings), plus the fixed cost of an engine and a compile. Replay time follows the work the run did before the pause. A snapshot wins by the ratio of the two, so it pays off for runs that compute a lot and keep little (Havlak, Richards, image-text: 200-650x), helps for render-like work (SSR 16x, markdown 6x), helps modestly for data held in large graphs (json-data 3x at both sizes), and loses or ties for short runs that pause early. That last group is what an agent-style program waiting on a tool usually looks like: its replay to the pause took 1-4 ms.

Size: the render and data heaps write 0.3-0.7 raw bytes per live heap byte, and four small AWFY heaps (Richards, DeltaBlue, Havlak, CD) 1.3-1.6. awfy-json writes 0.21, and lazy-builtins 0.03, because almost all it holds are built-ins, which are named rather than written. Strings are written inline at every reference and every object repeats its property keys, so data-heavy heaps compress very well (json-data: 7.3 MiB raw, 374 KiB zstd). A string table and shared shapes would shrink the raw format several fold.

The snapshot also costs the paused run: serialize plus the accounting check is under 20 ms for everything except the JSON workloads (130 ms at 4,000 orders, 510 ms at 16,000), paid at every pause whether or not the snapshot is ever used.

Test-Suite Census

With GOCCIA_HEAP_CENSUS=<dir>, the test runner records the live classes the file created at its 1st, 2nd, 4th, ... 64th await that suspends, after the module body (module-idle) and after its tests (tests-done). The census counts classes, not snapshot success: a class with a codec can still refuse (a native closure with an unregistered callback), and objects the run materialized from lazy built-ins (Temporal and Intl prototypes) are counted although a snapshot would name them. It is still a fair map of what real programs keep alive. To re-run:

./build.pas testrunner
GOCCIA_HEAP_CENSUS=/tmp/census ./build/GocciaTestRunner -P tests
bun scripts/heap-snapshot-measure.ts census /tmp/census

Every sampled await point holds a native closure the prototype cannot name (mostly the test runner's and the promise machinery's). The next classes without codecs, by the number of await-point samples they occur in: module namespace objects (83), error objects (79), test-runner expectations (67), async generators (36), async iterator records (30), AsyncLocalStorage (24), async context snapshots (23), async-from-sync iterators (17), fetch Headers and Response (13 each), dynamic import state (12), deferred module namespaces (6), SharedArrayBuffer (5) and Temporal values. Across all samples, 117 distinct classes were alive; the prototype has codecs for 21.

Coverage

Effort Estimate to Production Quality

Effort is counted in agent hours: the active time of one agent session, with idle gaps over 10 minutes (stalls, waiting for input) removed. Helper agents an agent starts in parallel are counted separately.

The prototype took about 1.2 agent hours across two sessions:

The estimate below starts from the rate measured in the second session, at prototype quality: about 5 minutes per codec for a class with plain fields (Set, the array iterator), 5-10 minutes when it has cross-object state (Promise.all), and 15-30 minutes for a subsystem (the lazy built-in table and its resolver). That rate already includes builds, failed builds and debugging. Production quality multiplies it by about 2.5: a test or fixture per codec (about +50%), a second agent's review and the fixes it asks for (about +50%), the edge cases the prototype refuses instead of handling (about +25%), and pull request and CI overhead (about +25%). Runs that are long by nature (fuzzing, GC-stress, the round-trip lane) are added at their own length.

The total is about 30 agent hours, 25 times the prototype, with a range of 20-45: a multiplier between 2 and 4 moves the coded pieces by about ±8 hours, and item 13 is where the long tail surfaces, its size unknown until the lane runs. Pieces 4-10 are independent of each other, so parallel sessions could bring the elapsed time to roughly 12-15 hours. Beyond the one-time work, every built-in class added later needs a codec or an explicit refusal, and every change to a codec-covered class's fields needs its codec and schema version updated: a standing cost of roughly 10-20% on built-in work.

A coverage-gated first version is about 13 agent hours (range 9-20): items 1, 2, 3 and 11, half of 12 and 13, the promise combinators, the built-in iterators, error objects and async generators, and a smaller documentation and review share. It covers single-module programs on the core language and refuses everything else, which the embedder answers by replaying.

Risks

• Silent divergence. A codec that forgets a field restores a run that behaves differently instead of refusing. The accounting check catches unserialized objects, not unserialized fields; only the round-trip lane catches those, and only for the states the tests reach.
• Trust. A snapshot encodes VM state such as instruction pointers and register counts. The reader bounds-checks, but a crafted snapshot can still describe a state the VM never produces. Snapshots must come only from the engine, authenticated by the embedder, and never from the script or a user.
• Coupling. Codecs read private fields of about a hundred classes in a codebase that changes daily, and the engine's lazy built-ins and cached intrinsics make built-in identity subtle (two prototype bugs came from there).
• FPC. Generic specializations in different units have different VMTs; interface changes in base units needed --clean builds several times during the prototype.
• Payoff. For short I/O-bound runs, the common case for a run waiting on a tool, replay is already as fast as a restore, so the work may buy little for the workloads that pause most.
• Determinism. Replay needs a deterministic run; a snapshot does not, but a restored run differs from an uninterrupted one wherever state lives outside the heap (random state, clocks) unless that state is captured too.

Recommendation

Do not productize heap snapshots now. Replay stays the guarantee, and the measurements say a snapshot is worth having only when the run did much more work before its pause than it keeps alive. Before deciding:

1. Record, in the embedder, how long resumed runs take to replay to their pause point.
2. If a meaningful share replays for more than about 100 ms, where a snapshot measured at least 5x faster, build the coverage-gated first version (about 13 agent hours) behind an engine option, with the round-trip lane from the start and replay as the answer to every refusal.
3. Extend coverage by what refusals in real traffic show, not by the full list above.

Keep the branch as the reference implementation: the lazy built-in table and path resolution, the idle-scratch release and the census are useful on their own if the work resumes.