buildprof reveals why Bun's Zig build was 5× slower than Rust build

Quick takeaway

buildprof proves that Bun’s Zig build was slow because a massive Full‑LTO linking phase and a single‑module Zig compilation prevented parallelism, whereas the Rust build used Thin‑LTO and many crates, cutting total time from ~24 min to ~5 min.

What is buildprof?

buildprof is an open‑source Linux tracing tool (https://github.com/lalitMaganti/buildprof) that records every process a build spawns, along with file reads/writes and optional compiler‑internal traces. It renders a hierarchical timeline where time flows left‑to‑right, bar width shows duration, and child processes appear beneath their parents. Use it by prefixing any build command:

buildprof -- make -j16
buildprof -- cargo build
buildprof -- ninja -C out/target
buildprof -- just build
buildprof -- ./dev/custom-build-script.sh

The recorded data can be explored in a web UI (https://buildprof.lalitm.com/) built on the Perfetto platform.

Why investigate Bun’s compile times?

A tweet by Jarred Sumner (Bun chief architect) claimed that Bun 1.4.0 (Rust‑based) compiled 5× faster on Linux than Bun 1.3.14 (Zig‑based). The tweet noted that the Zig build used Full LTO while the Rust build used ThinLTO. Since LTO strategy can dramatically affect build time, the author built buildprof to verify the claim and understand the underlying causes.

Reproducing the numbers

The author checked out Bun 1.3.14 and 1.4.0, scripted their CI builds on a 6‑core/12‑thread VM, and obtained timings close to the tweet:

Build Reported CI median Reproduced on single VM
Zig era (Bun 1.3.14) 30 m 06 s 24 m 24 s
Rust era (Bun 1.4.0) 5 m 37 s 5 m 40 s

The gap persisted, prompting a deeper probe.

Visualizing builds as process trees

A build is essentially a process tree: the top‑level command (e.g., cargo) spawns compilers, linkers, scripts, which may spawn further processes. By recording each fork/exec/exit event with ptrace and intercepting file I/O via a seccomp filter, buildprof constructs a complete tree and maps file dependencies between nodes. This approach is build‑system agnostic and automatically includes custom scripts.

The Zig build: a linker bottleneck

The timeline for the Zig‑era CI build shows a single ld.lld invocation consuming ~16 minutes (≈ 2/3 of total time). Enabling --compiler‑traces reveals internal LLD phases; the OptModule phase alone takes >10 minutes, confirming that Full LTO forces the linker to run heavyweight optimization passes over the whole program.

The Rust build: thin linking

The Rust‑era CI build’s timeline shows a total link time of 2 m 24 s. Its linker command includes -plugin-opt=thinlto, meaning the linker only performs lightweight symbol resolution while most optimization occurs earlier in parallel rustc invocations. This stark contrast points to LTO strategy as the primary cause of the time gap.

Experiment: switching Zig to ThinLTO

The author patched Bun’s Zig build to use ThinLTO and recorded a new run. The link time improved by 3 m 40 s, but the overall build still took ~13 minutes because many WebKit libraries (e.g., libJavaScriptCore.a) were still built with Full LTO. Those pre‑built archives forced the linker to perform full optimizations on large bitcode modules.

Rebuilding WebKit with ThinLTO

By rebuilding the required WebKit revision and its ICU dependencies with ThinLTO, and swapping the downloaded archives, the author achieved the following results:

Configuration Whole build Final linker
Original Full LTO (Zig) 24 m 24 s 16 m 35 s
Zig ThinLTO + original WebKit 20 m 20 s 12 m 55 s
Zig ThinLTO + rebuilt ThinLTO WebKit 15 m 11 s 7 m 22 s

The rebuilt WebKit cut the link time in half, but the build remained slower than the Rust version.

Parallelism: crates vs a single module

Inspecting the dependency arrows shows the linker waiting for bun-zig.o while the C++ side finishes earlier. The crucial observation: Bun’s Rust build is split into >90 crates, allowing Cargo to launch many rustc processes in parallel. In contrast, the Zig build compiles the entire codebase as one monolithic Zig module, limiting parallelism and inflating the linker’s workload (a single huge ThinLTO bitcode module vs many small ones).

Other findings

  • Miscellaneous CI steps – the trace captured network probes, Docker inspections, and Git queries, each sub‑second.
  • Cold dependency fetch – a fresh WebKit download/extraction took ~20 s, visible as node → tar → gzip events.
  • C++ compilation detail – enabling -ftime-trace for Clang showed a 12‑second compile split evenly between frontend and backend, with ModuleInlinerWrapperPass consuming >4 s.

How buildprof works

  • Recording – uses ptrace to follow forks/execs and a seccomp filter for file events. This avoids the permissions and stability issues of eBPF or ftrace.
  • Overhead – mainly proportional to the number of file opens. Example overheads: ripgrep (≈ 0.2 s), Redis (≈ 5 s). The --no-file-events flag can eliminate file‑tracing overhead.
  • UI – built on the Perfetto UI via its plugin system, providing timeline rendering, process tree layout, and on‑demand arrows between producers and consumers.

Related tools and why they fell short

Tool Strength Limitation
ninjatracing Shows Ninja edges and parallelism Misses wrapper scripts and subprocess trees
Cargo timings Detailed Cargo phase times Ignores custom scripts and non‑Cargo work
clang -ftime-trace Compiler‑internal phases No view of whole‑build process
strace / tracexec Generic system‑call tracing Not build‑oriented, hard to read timeline
What the Fork Process‑tree view Private beta, not open source

buildprof fills the gap by combining full process‑tree capture, file‑dependency arrows, and optional compiler traces in a single, interactive timeline.

Future directions

  • Reduce filesystem‑tracing overhead (especially for builds that open many files).
  • Add macOS and Windows support.
  • Support more build systems (npm, Gradle, Bazel).
  • Compute and display critical paths automatically.
  • Incorporate user feedback via GitHub issues.

Community reaction (selected HN comments)

"Good write up! I was hoping it was going to conclude with the author getting Bun's zig build way faster than Rust's but a deep dive nevertheless" – @anaqin

"It's like Electric Insight… you can diff builds to see why one is slower" – @t43562

"Nice! What would the macOS equivalent be?" – @jiehong

"Full LTO on the WebKit link is the serial part. Does your visualizer split link time from codegen?" – @xcc3641

These comments highlight interest in the tool’s applicability beyond this case study and curiosity about cross‑platform support.

Conclusion

  • The dominant factor in Bun’s Zig build slowdown was a Full‑LTO linker step that ran alone for ~16 minutes.
  • Switching to ThinLTO reduced link time but still left a large gap because pre‑built WebKit archives were still Full‑LTO.
  • Rebuilding WebKit with ThinLTO cut total build time to 15 minutes, yet the Rust build remained faster due to massive parallelism across >90 crates.
  • buildprof proved invaluable for exposing hidden bottlenecks, confirming hypotheses, and guiding concrete optimizations.

If you face slow builds, give buildprof a try: buildprof -- <your‑build‑command> and explore the timeline at https://buildprof.lalitm.com/.

Sources

Related

  • Project
    sopaco/deepwiki-rs

    Litho (deepwiki‑rs) is a Rust‑based tool that automatically generates C4‑style architecture documentation from a codebase using LLMs. It scans source files, extracts structure and comments, runs AI agents to infer architecture, creates Mermaid diagrams, and outputs a synced wiki. Supports many languages, can integrate external docs, and includes companion tools (Litho Book viewer, Mermaid Fixer). Designed for CI/CD integration and aims to eliminate manual documentation upkeep.

  • Dispatch
    Benchmarking Text Generation Inference

    Hugging Face introduces a benchmarking tool for Text Generation Inference (TGI) to help developers profile throughput and latency trade-offs to optimize LLM deployment costs and performance.

  • Dispatch
    LFM2.5-350M GRPO Fine-tuning Boosts IFStruct Score to 29.7%

    Fine‑tuning the 350M‑parameter LFM2.5 model with Group Relative Policy Optimization (GRPO) for just 100 steps raises its IFStruct benchmark score from 22.6% to 29.7%, demonstrating that inexpensive task‑specific reward training can markedly improve structured‑output compliance.

  • Dispatch
    Profiling in PyTorch: A Beginner's Guide to torch.profiler

    Hugging Face provides a comprehensive guide to using torch.profiler to identify bottlenecks, understand the CPU-GPU dispatch chain, and analyze the impact of torch.compile on kernel execution.

  • Project
    trailhq/Graft

    Graft is an open‑source TypeScript/Node.js tool that builds a local, markdown‑based knowledge graph of a codebase (summaries, key snippets, typed links) using tree‑sitter for structural parsing and optional LLM summarisation. The graph is cached in a `graft/` folder, refreshed only on file changes, and automatically wired into coding agents like Claude Code, Cursor, Codex, and Gemini. Benchmarks claim up to 46 % fewer tool calls, 42 % token savings, 60 % latency reduction, and a 12‑point correctness boost on SWE‑bench. Integration is via a single `graft init` command that writes agent‑specific instruction files; the CLI also offers search, visualization, and telemetry controls. The project is MIT‑licensed and targets faster, cheaper AI‑assisted development.