Browse documentation

Web Tooling Goccia benchmark lane

Date: 2026-07-06 Area: benchmarking, tooling, ci Issue: #857 Related: #821, ADR 0087

GocciaScript will integrate the V8 Web Tooling Benchmark as a Goccia-only external workload lane, not as a Node/QuickJS comparison lane. The suite answers whether GocciaScript can build and execute real JavaScript tooling bundles, so reference engines are unnecessary signal and would make CI spend time measuring the wrong thing. Node remains in the workflow only as the upstream package/build runtime for Webpack.

The upstream corpus is pinned by repository SHA in perf/web-tooling/manifest.json. CI runs every pinned upstream workload, not a curated subset: the current report set is acorn, babel, babel-minify, babylon, buble, chai, coffeescript, espree, esprima, jshint, lebab, postcss, prepack, prettier, source-map, terser, typescript, and uglify-js.

The driver prepares upstream's generated Terser/UglifyJS self-bundles, then generates a static entry and payload-only fs.readFile/fs.readFileSync adapter for each workload. The entry imports exactly one upstream *-benchmark module and times one direct call to its exported fn(); process repetitions are the raw samples. This keeps the upstream workload and third_party inputs intact while excluding Benchmark.js, Lodash, virtualfs, and unrelated workloads from the measured bundle. It also avoids making those measurement dependencies compatibility requirements for GocciaScript.

Each workload runs with GocciaScriptLoader in bytecode mode. The runner passes the broad ECMAScript compatibility flags needed by legacy tooling bundles, including var, function, ASI, non-strict Script semantics, legacy loops, loose equality, labels, arguments, and the explicit --unsafe-function-constructor gate. Those flags are part of the measurement contract: Web Tooling is a compatibility and viability probe, not a recommendation for default-style GocciaScript code.

Per-workload resource ceilings are also part of the manifest contract. PostCSS uses a 25 MB managed-heap ceiling so bytecode memory-pressure collections run before a constrained CI host can terminate the process.

The report is normalized JSON with one target entry per workload. A build failure, timeout, crash, OOM, syntax error, or missing runs/s line is recorded as a first-class target outcome so the report stays structurally complete and diagnosable. Those outcomes also gate the job: web-tooling-ci-report.js exits nonzero when a workload's build outcome is not ok or when any sample outcome is not ok. Correctness regressions — a workload that stops parsing or running — therefore fail CI instead of being visible only in the PR comment. Performance stays non-gating: no runs/s threshold is enforced, and a workload that runs to completion passes however slow it is. The job also fails when the driver cannot produce a structurally complete report for every manifest workload. CI runs one workload per matrix job, then validates and merges those shards into one report. PR CI uploads the web-tooling-report artifact and posts a Goccia-only comment. Main CI uploads the same artifact and, when BLOB_READ_WRITE_TOKEN is configured, publishes the compressed report plus a UTC daily pointer to Vercel Blob under web-tooling/.

Consequences:

  • Issue #857 is complete when every pinned Web Tooling workload has a Goccia report entry with build/execution outcome, raw sample data, and reproducible metadata.
  • Web Tooling results are retained like test262 and performance artifacts: workflow artifacts for every CI run, Vercel Blob run/daily records on main.
  • AWFY remains the cross-engine external benchmark lane. Web Tooling remains Goccia-only unless a future issue asks a different question.