inclusionAI/AReno

What it solves

AReno is a self-contained toolkit designed to make Large Language Model (LLM) reinforcement learning (RL) and post-training accessible on a single node. It eliminates the need for complex cluster setups or the manual wiring of separate training frameworks, inference servers, and kernel libraries, allowing developers to go from a base checkpoint to a served model locally.

How it works

AReno provides a full-stack design that integrates training, serving, and rollout capabilities into one package. It uses a Trainer class to manage the RL loop: generating on-policy completions (rollout), scoring them via a user-defined reward function, and updating model weights (training). It supports multiple hardware backends including NVIDIA CUDA, Ascend NPU, and Apple Silicon (via MLX) and offers an OpenAI-compatible proxy for agentic RL trajectories.

Who it’s for

Researchers and developers who want to perform fast, local LLM post-training, SFT/DPO-style training, or agentic RL without the overhead of managing a distributed cluster.

Highlights

• Plug-and-play: Access various post-training methods via a simple CLI flag or Python API.
• Multi-backend support: Native compatibility with CUDA, Ascend NPU, and Apple Silicon MLX.
• Agentic RL ready: Supports training from trajectories generated by agents interacting with a local OpenAI-compatible proxy.
• Multimodal capabilities: Handles image, audio, and video content using OpenAI-style message formats.
• LORA integration: Native support for training TP-aware adapters for models like Qwen3 and Bailing-MoE V3, saving standard PEFT artifacts.
• Extensible architecture: Allows easy registration of new algorithms, reward functions, and hardware backends.