VeRL-Omni v0.2.0 release notes / what's new

VeRL-Omni v0.2.0 establishes a production-grade foundation for omni-modal reinforcement learning (RL), focusing on increasing throughput for diffusion RL and stabilizing the training stack for omni-modal models. The release is centered on two primary advancements: the implementation of request-level batching for diffusion RL and the introduction of a reusable omni training architecture.

Faster Diffusion RL via Request-Level Batching

VeRL-Omni v0.2.0 significantly reduces rollout latency for diffusion RL by moving away from serial execution. In previous versions, diffusion rollouts were often bottlenecked by serial DiT forwards, leading to suboptimal GPU utilization.

Technical Improvements

  • Request-Level Batching: vLLM-Omni now packs compatible requests into larger transformer forwards instead of processing them through a serial loop. This becomes the default rollout path for supported diffusion adapters, allowing for explicit concurrency control via scheduling knobs.
  • V1 Trainer Integration: Diffusion RL now utilizes a V1 trainer path, aligning it with the modern trainer architecture used across VeRL-Omni and enabling the future decoupling of rollout and training execution.
  • Correctness Fixes: The update addresses critical areas to ensure policy consistency, including request-batched diffusion log-probs, async rollout semantics, rank-local LoRA weight-update routes, and rollout-correction hooks.

Performance Benchmarks

Using the Qwen-Image LoRA OCR recipe, the transition to request-level packing increased GPU utilization from approximately 80% to 100%. This resulted in a 52% reduction in isolated generation time, dropping from 226 seconds to 108 seconds.

For Qwen-Image LoRA with True-CFG at 512 px, the recommended max_num_seqs tuning range is 8 to 32. SD3.5 supports higher concurrency with a max_num_seqs of up to 256 due to a lighter memory footprint.

Stable Omni Training Architecture

VeRL-Omni v0.2.0 shifts from model-specific integrations to a reusable omni training stack. This allows multimodal autoregressive training to be integrated more naturally into the existing trainer, adapter, and rollout structures.

The Omni Training Stack

  • V1 Trainer Architecture: Omni recipes now benefit from clearer worker orchestration and standard configuration overrides, improving alignment with vLLM-Omni rollouts.
  • Reusable Omni Model Adapter: A shared interface (OmniModelBase) handles model setup, processor configuration, trainable-stage selection, FSDP preparation, and rollout alignment. This removes the need for one-off wiring for every new architecture.
  • Modular Call Flow: The main_omni.py entry point directs online omni jobs to the verl PPO V1 path. The PPO trainer manages the RL loop (rollout scheduling, advantage computation, and policy updates), while the specific OmniModelBase adapter (e.g., Qwen3OmniThinkerAdapter) handles model-specific logic such as stripping inactive modules and redirecting forwards to the target component.

MMK12 Benchmark Results

The stability of the new path is demonstrated via the MMK12 anchor recipe, which trains Qwen3-Omni on K12 visual math reasoning (image-to-text) using GSPO and LoRA (rank 32) on 4 x H800 80GB GPUs. The setup achieved:

  • Validation Reward: 0.833
  • Actor-Rollout Pearson Correlation: 0.998
  • GPU Memory Usage: Approximately 59 GB

Expanded Model and Algorithm Support

VeRL-Omni v0.2.0 expands its support matrix for various diffusion and omni-modal models:

Model / Family Modality Algorithm / Recipe Key Update
LTX2.3 Text $\to$ Video + Audio FlowGRPO Added text-to-video+audio training with CLAP and ImageBind rewards.
Qwen-Image-Edit Text + Image $\to$ Image FlowGRPO Added image-editing data preparation and training interface.
BAGEL Text + Image FlowGRPO Added full-parameter and LoRA recipes with OCR and PickScore rewards.
SD3.5 + DiNa-LRM Text $\to$ Image FlowGRPO Implemented latent reward model to score clean diffusion latents, bypassing VAE decode.
Flow-DPPO Text/Image $\to$ Image Flow-DPPO New policy-optimization recipe for Qwen-Image style diffusion RL.
Wan2.2 Text $\to$ Video DanceGRPO Added video-generation RL recipe coverage.

Additionally, the release includes Ascend NPU Dockerfiles and installation guidance.

Future Roadmap

Upcoming developments for VeRL-Omni include:

  • Implementing fully async training to optimize omni-modal models.
  • Extending support to models such as MiniMax-H3 and MiniCPM-o, and trainers like OPD/M-OPD.
  • Improving video diffusion training efficiency through batching, TQ, and V1 trainer integration.
  • Hardening rollout code for async training in both diffusion and omni-modal contexts.
  • Developing support for agentic RL involving multi-turn and multi-stage generation.

Sources

Related

  • Dispatch
    VeRL-Omni: RL Training Framework for Diffusion and Omni-Modality Models

    vLLM has announced the pre-release of VeRL-Omni, a general reinforcement learning post-training framework designed specifically for multimodal generative models, including diffusion and omni-modality architectures.

  • Project
    verl-project/verl-omni

    A general RL training framework designed for multimodal generative models, providing fast and stable post-training for diffusion and omni-modality models.

  • Dispatch
    vLLM-Omni Accelerates Inference with AutoRound Quantization

    vLLM-Omni now integrates Intel's AutoRound post-training quantization, enabling W4A16 quantization that cuts model size up to 62% while preserving accuracy and unlocking performance gains on Intel XPU and NVIDIA GPUs.

  • Dispatch
    vLLM Native RL APIs Release

    vLLM has introduced native weight syncing APIs and improved asynchronous RL support to standardize weight transfer between training and inference and eliminate deadlocks in large-scale DPEP deployments.

  • Dispatch
    vLLM-Omni Optimizations for Qwen3-Omni-30B-A3B-Instruct Serving

    vLLM-Omni serves Qwen3-Omni-30B-A3B-Instruct via a three-stage pipeline (Thinker, Talker, Code2Wav) and improves throughput and latency using stage decomposition, CUDA Graphs, async chunk handoffs, async output, stage replicas, and hot‑path cleanup.