inclusionAI/AReno

An easy-to-use, fast toolkit to scale up RL post-training on a single node.

What it solves

AReno is designed to make LLM reinforcement learning (RL) and post-training accessible by providing a self-contained, single-node toolkit. It eliminates the need for complex cluster setups, separate inference servers, or fragmented training frameworks, allowing developers to go from a base model checkpoint to a trained and served model on one machine.

How it works

AReno uses a full-stack design optimized for single-node performance. It provides a Trainer class and a CLI that handle the entire RL loop: generating on-policy completions (rollout), scoring them via user-defined reward functions, and updating model weights (training). It supports both CUDA (Linux) and MLX (Apple Silicon) backends natively.

Who it’s for

It is built for researchers and developers who want to perform fast, local post-training of LLMs using RL, SFT, or DPO-style training without the overhead of managing large-scale infrastructure.

Highlights

  • Plug-and-play: Access various post-training algorithms via a simple --algo flag or Python API.
  • Agentic RL Ready: Supports training agents by interacting with a local OpenAI-compatible proxy and training from trajectories.
  • Multimodal Support: Compatible with image, audio, and video content using OpenAI-style message formats.
  • Hardware Flexibility: Native support for NVIDIA GPUs via CUDA and Apple Silicon via MLX.
  • Integrated Serving: Includes a built-in OpenAI-compatible server with continuous batching for immediate deployment of trained models.
  • Operations Agent: Features a built-in AI assistant (areno agent) to help users configure and run training commands.

Related

  • Project
    Tencent-Hunyuan/UniRL

    UniRL is a reinforcement learning framework that provides a unified post-training loop for multimodal models, including image and video diffusion, autoregressive VLMs, and LLMs.

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.

  • Project
    OpenPipe/ART

    An open-source RL framework that uses GRPO to train multi-step AI agents to learn from experience and improve reliability in real-world tasks.

  • Project
    rllm-org/rllm

    An open-source framework for training language agents with reinforcement learning, allowing the same agent code to be used across different backends, sandboxes, and benchmarks.

  • Project
    facebookresearch/ReAgent

    An end-to-end platform for applied reinforcement learning designed for large-scale distributed tasks and offline training without a simulator.