beehive-lab/TornadoVM

Write Java. Run on GPUs. Fast.

What it solves

TornadoVM allows Java developers to run computationally intensive tasks on GPUs and multi-core CPUs without needing to write native C/C++ or CUDA code. It removes the complexity of managing JNI bindings and native toolchains, enabling high-performance computing (HPC) and AI workloads to be executed directly from the JVM.

How it works

It JIT-compiles Java bytecode into target-specific languages like NVIDIA CUDA, OpenCL C, and Apple Metal (MSL) at runtime. It provides three levels of API control:

  • Loop Parallel API: Uses @Parallel annotations to automatically infer thread mapping.
  • KernelContext API: Provides explicit thread indexing similar to CUDA/OpenCL for fine-grained control.
  • TileContext API: A specialized CUDA path that allows programming in tiles rather than threads, utilizing the tile compiler to optimize tensor-core instructions.

On NVIDIA hardware, it integrates directly with the NVIDIA library ecosystem (cuBLAS, cuFFT, cuDNN, cuDF) and exposes Tensor Core mma.sync intrinsics, allowing these native calls to share device buffers within a single TaskGraph.

Who it’s for

Java developers who need to accelerate machine learning, deep learning, computer vision, physics simulations, or financial applications using GPUs (NVIDIA, AMD, Intel, Apple Silicon) and multi-core CPUs.

Highlights

  • Multi-Backend Support: Works across NVIDIA (CUDA), AMD, Intel, and Apple Silicon (Metal) GPUs.
  • Native NVIDIA Integration: Direct access to cuBLAS, cuFFT, cuDNN, and cuDF without JNI bindings.
  • Tensor Core Access: Exposes mma.sync instructions and Tile Programming via the TileContext API.
  • CUDA Graph Support: Records and replays execution plans as a single CUDA Graph to reduce launch overhead.
  • JVM Compatibility: Plugs into existing OpenJDK or GraalVM installations without replacing the JVM.

Related

  • Project
    tensorflow/java

    TensorFlow‑Java provides Maven‑publishable Java/JVM bindings for TensorFlow, including low‑level JNI wrappers (`tensorflow‑core`), a high‑level neural‑network API (`tensorflow‑framework`), and an independent ndarray library. It supports Linux (x86_64, arm64) and macOS (Apple Silicon) binaries, with optional GPU builds, and integrates with standard Java build tools.

  • Project
    deeplearning4j/deeplearning4j

    A comprehensive JVM-based deep learning ecosystem that enables building, training, and deploying neural networks using Java, Scala, Kotlin, and Clojure.

  • Project
    deeplearning4j/deeplearning4j-examples

    A set of Maven‑based Java examples that demonstrate how to use the Eclipse Deeplearning4J ecosystem (DL4J, ND4J, SameDiff, DataVec, RL4J, etc.) for tasks ranging from basic neural‑network training to Spark‑distributed and GPU‑accelerated learning, plus model import and Android deployment.

  • Project
    lofcz/LLMTornado

    A provider-agnostic .NET SDK for building AI agents and workflows with built-in connectors to 30+ API providers and vector databases.

  • Dispatch
    Building a High-Performance LLM Inference Engine from Scratch with tiny-vLLM

    An exploration of tiny-vLLM, a C++ and CUDA-based educational project that teaches the internals of LLM inference, from Safetensors loading to PagedAttention.