LUPINE – GPU‑over‑IP Bridge

What it does – LUPINE lets a machine that has only a CPU (no GPU) run CUDA‑based programs by forwarding the GPU driver calls over the network to another machine that actually has a GPU. The remote GPU appears locally as if its driver libraries (libcuda.so, libnvidia‑ml.so, etc.) were installed on the client.

How you use it

1. Run the server on a GPU‑equipped host (Docker image lupine-server). It listens on TCP port 14833 and exposes the GPU via an HTTP/2 RPC interface.
2. Run the client on the CPU‑only host (Docker image lupine-client). Set the environment variable LUPINE_SERVER=<host>:14833. The client pre‑loads the LUPINE shim libraries, so any CUDA program (e.g., nvidia‑smi, PyTorch, custom kernels) automatically talks to the remote GPU.
3. Optional extras – enable tracing (LUPINE_TRACE), TLS (front the server with an HTTPS proxy), or multi‑GPU across several servers by providing a comma‑separated list of LUPINE_SERVER endpoints.

Key technical points

• Driver‑only forwarding – Only the low‑level driver calls cross the wire; the application still brings its own CUDA runtime.
• Shims – LUPINE ships thin “shim” libraries (libcuda.so.1, libnvidia‑ml.so.1, NCCL, nvSHMEM) that translate driver calls into RPCs.
• Compression & keep‑alive – HTTP/2 bodies are Zstd‑compressed; TCP keep‑alive (60 s idle, 15 s probes) prevents idle‑flow timeouts.
• Metrics – /metrics endpoint exposes Prometheus‑compatible GPU usage stats without extra monitoring agents.
• Graceful shutdown – SIGTERM drains in‑flight CUDA calls and optionally runs a checkpoint provider supplied via LUPINE_CHECKPOINT_LIBRARY.
• Multi‑GPU / multi‑server – Devices from several servers are presented as a single ordinal list; cross‑server cuMemcpyPeer works by staging through the client.
• Platform support – Pre‑built client bundles for Linux, macOS, Windows (amd64 & arm64). Linux clients target the manylinux_2_28 ABI (glibc ≥ 2.28).

Typical workflow

# GPU host
docker run --rm --gpus all -p 14833:14833 \
  ghcr.io/lupinemachines/lupine-server:cuda-13.3.1-ubuntu24.04

# CPU‑only host – run any CUDA program
docker run --rm -it -e LUPINE_SERVER=<gpu‑host>:14833 \
  ghcr.io/lupinemachines/lupine-client:cuda-13.3.1-ubuntu24.04 nvidia-smi

You can also build a PyTorch‑based client image (Dockerfile shown in the README) and run a tiny training job (microgpt_train) against the remote GPU.

Who it’s for – Researchers or engineers who have access to powerful GPUs in a separate data‑center or on a cloud VM but want to develop on a lightweight CPU‑only workstation without installing the full CUDA driver stack locally.

Limitations

• No built‑in authentication; you must place the server behind a TLS‑terminating proxy if security is needed.
• Cross‑server peer‑to‑peer copies are staged through the client, which adds extra host‑memory traffic.
• Older Linux distributions (CentOS 7, Ubuntu 18.04) are no longer supported.

Where to learn more – The repository contains detailed docs on checkpoint providers, tracing, and how to compile the shims locally. Prometheus metrics are available at http://<server>:14833/metrics.

---

All information above is taken directly from the project’s README.