Performance benchmarks
This page explains the benchmark metrics, methodology, and interpretation for the utoo package manager. Full data and historical runs are tracked in GitHub issue #3250 .
Lower times are better. Platform, registry, network topology, and filesystem behavior affect absolute timings, so results from different environments should not be mixed into one statistical comparison.
Metric definitions
| Metric | Name | State before install | What it measures |
|---|---|---|---|
p0 | Full cold | No lockfile, no manifest cache, no tarball/package store, no node_modules | First-install dependency resolution, network requests, download, extraction, and write cost |
p3 | Lock-only cold | Lockfile exists, but tarball/package store and node_modules are removed | Restoring dependencies from a locked graph when the package store is cold |
p4 | Warm | Lockfile and tarball/package store exist; only node_modules is removed | Rebuilding the project dependency directory from a populated global package store |
Do not use “no lockfile but warm manifest cache” as a comparison state. Bun keeps fresh manifests for a short period without revalidation, so that state gives Bun an extra advantage that does not match the intended cold-install semantics.
2026-08-27
registry.npmjs.org · GitHub Actions
- Harness branch/commit:
codex/pm-bench-pnpm12/2204fd0f4 - Project:
ant-design - Registry:
https://registry.npmjs.org - Workflow: GitHub Actions #33039114486
- Versions: utoo built from
2204fd0f4, Bun 1.4.0, pnpm 10.34.5, pnpm 12.0.0, aube 1.29.1 - Sampling: one measured round for
p0/p3; 5 interleaved rounds forp4. Treatp0/p3as directional smoke evidence only.
Linux (ubuntu-latest)
| Scenario | utoo | Bun | pnpm 10 | pnpm 12 | aube |
|---|---|---|---|---|---|
Full cold (p0) | 7.87s | 9.62s | 29.56s | 10.14s | 16.13s |
Lock-only cold (p3) | 9.66s | 10.40s | 18.32s | 10.58s | — |
Warm (p4) | 2.35s ± 0.04s | 3.65s ± 0.34s | 6.14s ± 0.07s | 3.39s ± 0.24s | — |
macOS (macos-latest, Apple Silicon)
| Scenario | utoo | Bun | pnpm 10 | pnpm 12 | aube |
|---|---|---|---|---|---|
Full cold (p0) | 20.46s | 23.42s | 57.19s | 30.32s | 51.44s |
Lock-only cold (p3) | 15.80s | 25.31s | 60.12s | 31.55s | — |
Warm (p4) | 4.76s ± 0.20s | 6.43s ± 0.42s | 24.69s ± 3.67s | 21.89s ± 2.63s | — |
In this smoke run, utoo was fastest in all three user-visible install states on both Linux and macOS. Aube completed p0, but its seed step did not produce the expected aube-lock.yaml, so the harness rejected its p3 / p4 cells instead of timing lockless or stale state.
registry.npmmirror.com · Linux China
- Harness branch/commit:
codex/pm-bench-pnpm12/2204fd0f4 - Project:
ant-designate8fbc23658a323812864d56d53b5d25b73784dd1 - Host: Linux China x86_64, 96 logical CPUs, 16 GiB RAM, Node.js 22.18.0
- Registry:
https://registry.npmmirror.com - Network: direct connection; proxy environment variables were unset and the benchmark command explicitly selected npmmirror
- Versions: utoo built from
2204fd0f4, Bun 1.3.14, pnpm 10.34.5, pnpm 12.0.0, aube 1.29.1 - Sampling: 3 interleaved rounds for
p0/p3; 5 interleaved rounds forp4
| Scenario | utoo | Bun | pnpm 10 | pnpm 12 | aube |
|---|---|---|---|---|---|
Full cold (p0) | 12.04s ± 1.25s | 19.63s ± 1.32s | 33.88s ± 0.64s | 18.50s ± 6.22s | 26.47s ± 7.65s |
Lock-only cold (p3) | 22.28s ± 12.71s | 31.75s ± 15.11s | 40.70s ± 11.64s | 31.67s ± 10.09s | — |
Warm (p4) | 1.22s ± 0.07s | 4.58s ± 0.32s | 9.86s ± 0.48s | 2.60s ± 0.09s | — |
Interpretation:
p0: utoo has the fastest mean. pnpm 12 is about 54% slower by mean, though both should be interpreted alongside their raw samples.p3: utoo has the fastest mean and pnpm 12 is about 42% slower. Variance is high, so this is directional evidence only.p4: utoo takes 1.22s versus pnpm 12’s 2.60s. Both have low variance, making this the run’s strongest signal.- Aube’s
p3/p4cells were rejected because the expected lockfile was not produced, so they are not ranked.
Methodology notes
The phased harness uses an explicit store for each package manager, restores each package manager’s own lockfile, disables lifecycle scripts, and alternates package-manager order between rounds to reduce network and ordering bias.
The harness also records network traffic so accidentally cold warm cells can be rejected. For example, p4 should primarily reuse the existing lockfile and tarball/package store; if it still performs meaningful downloads, it is not a valid warm result.
The legacy pm-bench-all result should not be cited as a utoo-vs-Bun performance comparison. Its warm preparation deletes the generated lockfile while retaining global caches, so it measures fresh resolution with a warm manifest cache instead of the p4 state defined here.