NVIDIA Nemotron 3.5 Super VL Day 0 Support on vLLM: 120B MoE Multimodal Agent Model

TL;DR

vLLM shipped day 0 support for NVIDIA Nemotron 3.5 Super VL, an open, customizable multimodal model built for long-running agents. The model uses a 120-billion-parameter mixture-of-experts (MoE) architecture with 12 billion active parameters, accepts text, images, documents, and video as input, and outputs text, and it can be served through vLLM with continuous batching, prefix caching, speculative decoding, and an OpenAI-compatible API.

The announcement matters because it pairs a high-accuracy agentic execution model with a production serving stack on the same day the weights become available, including an NVFP4 quantized checkpoint and built-in multi-token prediction (MTP) heads for faster decoding.

What NVIDIA Nemotron 3.5 Super VL is

NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model built for long-running agents. It combines reasoning, instruction following, and tool use with text, image, document, and video understanding in a single model, which reduces the need to deploy separate text and vision models and helps maintain task-relevant context across workflow steps.

The model is built on a 120-billion-parameter mixture-of-experts architecture with 12 billion active parameters. According to NVIDIA, this design balances accuracy and efficiency for specialized, always-on agent tasks. Organizations can post-train the model with NVIDIA NeMo using their own data, tools, policies, and workflows.

Within a system of models, Nemotron 3.5 Super VL is positioned as the default high-accuracy agentic execution model with native document, image, and video understanding. It complements two other models in the same family: NVIDIA Nemotron 3.5 Lightning for high-volume text execution, and NVIDIA Nemotron 3 Ultra for the hardest text-only reasoning.

Key specifications

Attribute Value
Architecture 120B total parameters, 12B active parameters, mixture-of-experts
Modalities Text, images, documents, and video as input; text as output
Optimizations C-RADIOv4 vision encoder, hybrid Mamba-Transformer architecture, multi-token prediction, NVFP4 quantization
Model role High-accuracy agentic execution with native visual understanding in a system of models
Customization Post-training with NVIDIA NeMo using organization-specific data, tools, policies, and workflows
Deployment NVIDIA NIM for optimized production serving across cloud and data center environments
Deployment targets 1x GB10, B200, B300, RTX PRO 6000, H200, and 2x H100

The two requirements the model targets

NVIDIA frames Nemotron 3.5 Super VL around two practical requirements for multimodal, always-on agents:

  • Native visual understanding with agentic execution. The model combines high-accuracy reasoning and tool use with document, image, and video understanding, preserving task-relevant context across specialized workflows.
  • Efficient, controllable deployment. A hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization are designed to improve generation efficiency and create headroom for more concurrent long-context sessions.

How to serve the model on vLLM

vLLM provides the serving layer that brings Nemotron 3.5 Super VL into production workflows. The stack includes continuous batching, prefix caching, speculative decoding, and an OpenAI-compatible API. The BF16 checkpoint offers a straightforward baseline for deployment, and an NVFP4 checkpoint is also available at launch for environments that can take advantage of lower-precision inference.

Install vLLM

docker pull vllm/vllm-openai:v0.31.0

docker run --rm -it --gpus all --ipc=host --network=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /bin/bash \
  vllm/vllm-openai:v0.31.0

Serve the model

The published NVFP4 launch configuration is an example for a 1x GB200 setup, and the source advises validating it on the target hardware before use.

vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
  --served-model-name nemotron-3.5-super \
  --trust-remote-code \
  --max-model-len 114688 \
  --reasoning-parser nemotron_v3 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5

Once the server is running, applications can send prompts through an OpenAI-compatible client. The example below sends a video URL and reads both the thinking trace and the final answer from the response:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="null")

response = client.chat.completions.create(
    model="nemotron-3.5-super",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe what happens in this video."},
                {
                    "type": "video_url",
                    "video_url": {"url": "https://example.com/clip.mp4"}
                },
            ],
        }
    ],
    temperature=1.0,
    top_p=0.95,
    max_tokens=4096,
)

print(response.choices[0].message.reasoning)  # thinking trace
print(response.choices[0].message.content)    # final answer

Multi-token prediction for long-running agentic workflows

Nemotron 3.5 Super VL supports Multi-Token Prediction (MTP) to accelerate token generation while preserving the target model's output quality. The model includes built-in MTP heads: during decoding, these heads propose future tokens and the target model verifies them, reducing the number of sequential generation steps required for longer responses.

MTP can also be enabled with three speculative tokens instead of five:

vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
  --served-model-name nemotron-3.5-super \
  --trust-remote-code \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 3

Reported accuracy and efficiency

NVIDIA states that Nemotron 3.5 Super VL is designed to deliver high accuracy for specialized multimodal agent tasks while using only a subset of its total parameters for each token. The hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization are cited as the factors that accelerate output generation.

The published performance claims are:

  • On a single GPU, the model delivers up to 5x faster throughput for agentic tasks compared with other open models in its size category.
  • The model reaches over 600 output tokens per second at 100,000 input tokens, compared with other open models. This figure is measured on Artificial Analysis Intelligence Index agentic tasks on a Baseten endpoint.
  • On four B200 GPUs, the model performs tasks 4x faster compared with other leading open models of similar size, based on a chart comparing SWE-bench Pro accuracy against hours to complete 10,000 tasks. Nemotron 3.5 Super VL has the shortest completion time in that comparison.

One caveat in the source material: a third figure showing a DGX Spark comparison covers the related Nemotron 3.5 Super model and explicitly does not measure Nemotron 3.5 Super VL. In that chart, Nemotron 3.5 Super scores 80 on the local.ai intelligence index and takes 6.5 minutes per task on DGX Spark.

Example use cases

NVIDIA lists five target workloads for the model:

  • Customer service: Resolve multi-step requests by interpreting customer-provided documents and images, applying policy, and using service tools.
  • Document intelligence: Interpret complex documents, including OCR, tables, charts, and page layouts, to extract information and generate summaries or reports.
  • Enterprise productivity: Complete multi-step tasks by interpreting visual information, reasoning across it, and using tools.
  • Video understanding: Extract key information from video to support search, content review, advertising, retail, and incident analysis.
  • Web research: Search across websites, connect information from multiple sources, and verify results.

Availability

  • Download the model weights from Hugging Face in BF16 and NVFP4 formats.
  • Run Nemotron 3.5 Super VL with vLLM using the getting-started cookbook.

The vLLM blog post credits NVIDIA contributors Nirmal Kumar Juluru, Anusha Pant, Faradawn Yang, Pen Chung Li, Alex Steiner, Ran Zilberstein, Nave Assaf, and Amir Klein.

Sources