beehive-lab/TornadoVM

🌪️ TornadoVM – Java on GPUs

What it is – TornadoVM is an open‑source runtime that lets you write plain Java kernels and have them executed on GPUs (NVIDIA, AMD, Intel, Apple Silicon) or multi‑core CPUs. At runtime it JIT‑compiles the Java bytecode to the native GPU language of the target (CUDA PTX, OpenCL C, or Apple Metal MSL) and handles all host‑device data movement for you. On NVIDIA hardware it also calls directly into the CUDA library stack (cuBLAS, cuFFT, cuDNN) and even exposes Tensor‑Core mma.sync intrinsics, all from Java code.

🎯 Core ideas

Idea	How it works
Write once, run everywhere	Write a kernel in Java using a CUDA‑style thread index (`ctx.globalIdx`) or simply annotate a loop with `@Parallel`. TornadoVM builds a TaskGraph that can be dispatched to any of its back‑ends (OpenCL, CUDA, Metal, CPU).
JIT compilation	Java bytecode → Graal IR → backend‑specific source (CUDA PTX / OpenCL C / MSL) → runtime compiler (NVRTC, OpenCL driver, Metal) → native GPU binary, specialized for the actual data sizes.
Zero‑JNI plumbing	No hand‑written native wrappers are needed; the runtime generates the native code and links it to the Java process automatically.
Native CUDA ecosystem integration	Generated kernels share device buffers and a CUDA stream with library calls (`cublasSgemv`, FFT, cuDNN ops, Tensor‑Core intrinsics). You can mix custom Java kernels and vendor‑tuned library tasks in the same graph.
CUDA Graph capture	The whole graph (kernels, library calls, transfers) can be recorded once and replayed with a single `cuGraphLaunch`, giving launch‑time overhead comparable to hand‑written CUDA.

🚀 Typical use‑cases

Domain	Example projects
Large language‑model inference	`GPULlama3.java` runs Llama‑3, Qwen‑3, Mistral, etc. in pure Java at ~117 tokens / s on an RTX 5090; used by LangChain4j and Quarkus.
Real‑time graphics	`TornadoVM‑Ray‑Tracer` demonstrates interactive ray tracing written entirely in Java.
Computer vision / 3D reconstruction	`kfusion‑tornadovm` implements KinectFusion pipelines on GPUs.
Scientific / engineering codes	Linear‑algebra kernels (SGEMV/SGEMM), FFTs, physics simulations, financial modelling – all benefit from cuBLAS/cuFFT acceleration.
General data‑parallel workloads	Any embarrassingly parallel loop can be annotated with `@Parallel` and off‑loaded to GPUs or CPUs without rewriting the algorithm.

📦 Getting started (quick‑start)

Install – sdk install tornadovm (choose opencl, cuda, metal, or full).
Verify devices – tornado --devices.

Run an example –

java @$TORNADOVM_HOME/tornado-argfile \
     -cp $TORNADOVM_HOME/share/java/tornado/tornado-examples-5.2.0.jar \
     uk.ac.manchester.tornado.examples.compute.MatrixVectorRowMajor

Add to a Maven project –

<dependency>
  <groupId>io.github.beehive-lab</groupId>
  <artifactId>tornado-api</artifactId>
  <version>5.2.0-jdk21</version>
</dependency>

Write a kernel – either use the low‑level KernelContext API (shown in the README) or the higher‑level @Parallel annotation. Build a TaskGraph, optionally add library tasks (CuBlas::cublasSgemv, etc.), and execute with TornadoExecutionPlan.

🛠️ Main features

Multi‑backend: OpenCL, CUDA, Metal, plus a CPU fallback.
Native NVIDIA library support: cuBLAS/cuBLASLt, cuFFT, cuDNN (including flash‑attention), Tensor‑Core intrinsics.
TaskGraph abstraction – declarative data transfers, kernel launches, and library calls.
CUDA Graph capture for low‑overhead repeated execution.
Cross‑platform: works on Linux, macOS, Windows; supports integrated GPUs and FPGAs via OpenCL.
SDKMAN! distribution and Docker images for easy setup.
Apache 2.0 API + GPLv2‑CE runtime – permissive for commercial use.

👥 Community & support

Slack: https://join.slack.com/t/tornadovmcommunity/...
GitHub Discussions: https://github.com/beehive-lab/TornadoVM/discussions
Issue tracker: labelled “good first issue” for newcomers.
Documentation: ReadTheDocs site with programming guide, benchmarking guide, and API reference.
Academic & industry collaborations – part of NVIDIA Inception Program, funded by Intel, EU, UKRI, etc.

📜 License

Tornado‑API and most modules – Apache 2.0 (no copyleft on your code).
Runtime and drivers – GPLv2 with Classpath Exception (same as OpenJDK), also non‑copyleft for downstream applications.

Bottom line – TornadoVM brings the performance of native GPU code to the Java ecosystem without requiring you to write CUDA or OpenCL yourself. It is a real, production‑ready project that is already being used for LLM inference, ray tracing, and scientific computing.