lupinemachines/lupine

LUPINE is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

LUPINE – GPU‑over‑IP bridge

What it is – LUPINE lets a machine that has no GPU (or only a CPU) use a GPU that lives on another host over the network. It does this by providing a thin CUDA driver shim (libcuda.so.1 and libnvidia‑ml.so.1) that forwards all CUDA API calls to a remote server process via a persistent TCP connection.

Why it matters for AI/ML – Training and inference with modern deep‑learning frameworks (PyTorch, TensorFlow, etc.) require a CUDA‑compatible GPU. LUPINE makes those GPUs appear locally, so existing code can run unchanged on cheap CPU‑only workstations, CI runners, or even Macs, while the heavy compute stays on a remote server.

Core components

Component	Role
lupine‑server (Docker image)	Runs on a machine that has one or more GPUs. It receives RPCs from clients, executes the real CUDA calls, and returns results.
lupine‑client (Docker image)	A drop‑in replacement for the CUDA driver library. When a CUDA program starts, the client’s `libcuda.so.1` intercepts calls and forwards them to the server.
shims (`libcuda.so.1`, `libnvidia‑ml.so.1`)	Placed on the client’s `LD_LIBRARY_PATH` so tools like `nvidia‑smi` and any CUDA runtime library automatically use LUPINE without code changes.
checkpoint provider (optional)	A user‑supplied library that can persist a connection identifier and restore state after a graceful server shutdown.

Key features (as described in the README)

Docker‑first distribution – Server and client are published as ready‑to‑run containers for any CUDA version (e.g., cuda-13.1.0-ubuntu24.04).
Hosted demo – A public demo server with a Tesla T4 is available; you can try it with a single docker run command.
Cross‑platform – Works from Linux, macOS (via Docker), and any platform that can run the client container.
Multi‑GPU, multi‑server – You can list several LUPINE_SERVER endpoints; GPUs are exposed as a single ordinal list and cross‑host cuMemcpyDtoD/cuMemcpyPeer are handled by staging through the client.
Connection stability – TCP keep‑alive (60 s idle, 15 s probes, 3 retries) and exponential‑backoff connect retries keep long‑running training jobs from hanging when idle.
Trace logging – Set LUPINE_TRACE to 0/1/2 or a file path to get detailed client/server RPC traces.
Device printf forwarding – When a kernel uses printf, LUPINE captures the device buffer and forwards it to the client’s stdout.
TLS support via proxy – The server speaks plain HTTP/2; you can front it with any TLS‑terminating reverse proxy (HTTPS URLs are accepted).
Graceful shutdown checkpoints – On SIGTERM the server stops accepting new connections, lets in‑flight CUDA calls finish, and optionally invokes a user‑provided checkpoint library.

Typical workflow

Start the server on a GPU‑equipped host:

docker run --rm --gpus all -p 14833:14833 \
  ghcr.io/lupinemachines/lupine-server:cuda-13.1.0-ubuntu24.04

Run the client on the CPU‑only host, pointing it at the server:

docker run --rm -it -e LUPINE_SERVER=<server_ip>:14833 \
  ghcr.io/lupinemachines/lupine-client:cuda-13.1.0-ubuntu24.04 nvidia-smi

The output shows the remote GPU(s) as if they were local.

Run any CUDA program (PyTorch, TensorFlow, custom CUDA code) inside the client container. The program sees a device named lupine:0 (or lupine:1, …) and can use it normally.
Optional: multi‑GPU – Provide a comma‑separated list of servers in LUPINE_SERVER to aggregate GPUs from several machines.
Optional: checkpoint provider – Set LUPINE_CHECKPOINT_LIBRARY and LUPINE_SESSION if you need to preserve connections across server restarts.

Example use case – PyTorch training

The repo ships a small Dockerfile that builds a PyTorch‑enabled client image. After building it, you can run a training script (microgpt_train) against a remote RTX 4090 and see the loss decreasing, proving that a full training loop works over the network.

Limitations & gotchas (from the README)

Latency – The network replaces the PCIe link, so large data transfers (e.g., video encoding, massive host‑to‑device copies) can become a bottleneck. Model training usually moves the model once and then does mostly device‑side work, so the impact is modest.
No direct server‑to‑server peer‑access – Cross‑host cuMemcpyPeer is implemented by staging through the client; true zero‑copy peer‑access is not yet supported.
Authentication/TLS – LUPINE itself does not implement auth; you must place it behind a TLS‑terminating proxy or add your own front‑end if you need encryption or credential checks.
Requires matching CUDA versions – Client and server images must be built for the same CUDA major/minor version; otherwise the shims will not be compatible.
Checkpoint provider is optional – If you don’t supply one, graceful shutdown still works but no state is persisted.

Getting started quickly

# 1. Try the public demo (no setup needed)
docker run --rm -e LUPINE_SERVER=demo.lupinemachines.com:14833 \
  ghcr.io/lupinemachines/lupine-client:cuda-13.1.0-ubuntu24.04 nvidia-smi -L

# 2. Run your own server on a GPU box
docker run --rm --gpus all -p 14833:14833 \
  ghcr.io/lupinemachines/lupine-server:cuda-13.1.0-ubuntu24.04

# 3. From a CPU‑only box, run a CUDA program (e.g., PyTorch)
export LUPINE_SERVER=$(hostname -I | awk '{print $1}'):14833
docker run --rm -e LUPINE_SERVER=$LUPINE_SERVER \
  ghcr.io/lupinemachines/lupine-client:cuda-13.1.0-ubuntu24.04 python3 -c "import torch; print(torch.cuda.is_available())"

If the last command prints True, LUPINE is correctly forwarding CUDA calls.

Where to look next

Source code – codegen/ contains the generator that creates the RPC stubs from CUDA headers; checkpoint_provider.h defines the optional checkpoint ABI.
Documentation – The README covers most operational knobs; for deeper integration (e.g., custom checkpoint libraries) read the header files in the repo.
Community – Issues and pull requests on the GitHub repository discuss adding TLS front‑ends, multi‑host peer‑access, and other enhancements.

Bottom line – LUPINE is a practical, open‑source way to turn any network‑accessible GPU into a drop‑in CUDA device for AI training and inference, with Docker images that make deployment straightforward.