Reducing vLLM startup time with uv bytecode compilation

Fresh vLLM containers can achieve substantial startup time reductions by shipping precompiled Python bytecode, which avoids repeated compilation during runtime imports.

By setting the UV_COMPILE_BYTECODE=1 environment flag before installing runtime packages, the uv build tool compiles Python source code into bytecode during the build process. This practice avoids the costly repeated compilation that occurs during imports shared across models, leading to measurable startup savings.

Performance Gains on H100 Servers

Shipping precompiled bytecode resulted in significant startup time savings when running on H100 servers, demonstrating consistent performance improvements across different model sizes.

For Qwen3-8B and Qwen3-32B, the median paired startup savings were 17.6 seconds and 17.8 seconds, respectively. These measurements were taken on a Xeon Platinum 8480+ / H100 SXM 80GB server, comparing packages with shipped bytecode against those with caches removed. It is important to note that these timings exclude image pull and model download times.

The percentage reduction in startup time was larger for the 8B model (17.6%) compared to the 32B model (13.9%), which is consistent with avoiding Python import work shared by both models.

Implementation and Official Support

Users can implement this performance improvement by setting the UV_COMPILE_BYTECODE=1 flag before installing runtime packages during their image build. This instructs uv to compile Python source files into bytecode during the build phase.

For users relying on official vLLM images, the necessary change has been included in the CUDA nightly build (e.g., vllm/vllm-openai:nightly-8cbd5d03006c33185f402249ff2b448efd594986). This feature will be natively supported in official release images starting with version v0.32.0, eliminating the need for users to manually set the build flag.

Technical Mechanism of Startup Reduction

Startup time is reduced because Python, by default, compiles source code into code objects before execution. A valid .pyc file stores this compiled code, allowing Python to load it directly upon subsequent imports, bypassing the compilation step.

Standard package managers like pip compile bytecode during installation, but uv skips this step by default. When bytecode is shipped in the image, the Python interpreter can load the precompiled files immediately. This significantly reduces the time spent in source-compiler calls during the serving-CLI import process.

In a diagnostic test, the serving-CLI import time dropped from 5,257 source-compiler calls to zero when using a precompiled image. Furthermore, in a standalone installation outside Docker, valid bytecode reduced the instrumented serving-CLI import from 12.20 seconds to 5.01 seconds, a saving of 7.19 seconds.

Cross-Hardware Performance Observations

Performance gains were also observed on consumer hardware, though the results are kept separate due to differing hardware and runtime environments.

On an i9-13900HX / RTX 4090 Laptop, matched-image studies involving Qwen3-0.6B, Qwen3-1.7B, and SmolLM2-1.7B showed median paired savings ranging from 12.8 to 13.5 seconds, favoring precompilation across all 18 tested pairs.

In a separate, non-Docker installation, precompiled package caches saved 9–12 seconds per server launch when the installed files were already resident in memory. This measurement confirmed the saving even when the files were kept in the OS page cache.

Implementation Caveats and Tradeoffs

While beneficial, enabling bytecode compilation introduces specific considerations for image building and deployment.

  • Image Size: The precompiled image adds approximately 153 MB compressed (or 1.8% of the total size in one test), which may be relevant during a slow initial image pull.
  • Build Process: When building custom images, the UV_COMPILE_BYTECODE=1 flag must be set before uv commands install runtime packages. Users must also verify the final image and use the runtime Python's compileall function to prepare any missing caches.
  • Scope of Savings: The primary saving comes from compiler time reduction during imports. Module initialization and other setup tasks remain, and the full server-startup saving is not solely attributable to the compiler.

For detailed environments, run orders, and controls, the full methods and measurements are available in the supplementary data.

Sources