NVIDIA Nemotron 3.5 Super VL: 120B Open Model Claims Up to 5x Faster AI Agent Throughput

NVIDIA has released Nemotron 3.5 Super VL, an open-weight multimodal AI model with 120 billion total parameters, designed for agents that read documents, analyze images and video, use tools and complete multi-step work. The October 15 launch comes with an eye-catching performance claim: up to five times the inference throughput of selected competing open models in its class. But the important detail is what NVIDIA measured, and how the model achieves those results.

What is NVIDIA Nemotron 3.5 Super VL?

Nemotron 3.5 Super VL is a vision-language and reasoning model built for organizations that want adaptable AI agents without relying entirely on proprietary cloud models. It accepts text, images and sampled video frames, then returns text, structured information or tool-use instructions as part of an application. Its intended use cases include document analysis, coding assistance, security triage, web research and video understanding.

It is an update to NVIDIA's Nemotron lineup, not simply a larger version of the original Nemotron 3 Super introduced earlier in 2026. The new 3.5 Super VL adds a strong multimodal focus and updated training and inference techniques.

120 billion parameters, but only 12 billion active: how does that work?

The model uses a mixture-of-experts (MoE) architecture. It contains about 120 billion parameters in total, but routes each token through a smaller set of specialized components, activating approximately 12 billion parameters at a time. This helps the model retain broad capacity without paying the full computational cost of activating all 120 billion parameters for every token.

Under the hood, NVIDIA combines Mamba-2 layers and Transformer attention with a vision encoder. The architecture supports a context window of up to one million tokens, subject to the chosen serving configuration and hardware. A long context can help an agent work with lengthy reports and histories, but it does not guarantee perfect recall or accuracy across every token.

Is Nemotron 3.5 Super VL really five times faster?

NVIDIA reports up to five times higher throughput than other open-weight models in the same size category under specified inference conditions. A chart in the company's technical announcement shows more than 600 output tokens per second on a Baseten endpoint for an Artificial Analysis test using a 100,000-token input. NVIDIA also reports faster end-to-end completion for a separate coding-task evaluation running on B200 GPUs.

Throughput is not the same as universal response speed. These figures do not establish that Nemotron beats GPT, Claude, Gemini or every competing model by five times in all tasks. Results change with GPU type, quantization, prompt length, concurrency, decoding setup and how an agent uses tools. The most meaningful production metric is often the cost and time required to complete a correct task, not raw tokens per second alone.

What can the model actually do?

Read documents and charts: extract information from tables, financial reports, page images and complex layouts, then connect evidence across sources.

Support coding agents: analyze errors, examine repositories, test hypotheses and work through multi-step tool workflows.

Understand images and video: answer questions about visual content, assist with indexing and review, and analyze incidents captured in footage.

Help enterprise agents: follow internal procedures and use tools across repeated tasks, including security analysis and financial-document processing.

The distinction matters: Super VL understands video but is not presented as a text-to-video generation model. Its documented output modality is text.

Benchmarks and real-world use cases

NVIDIA reports an 80.8% result on CharXiv, a chart-reasoning benchmark, compared with 63% for Nemotron 3 Nano Omni. On NVIDIA's published Hermes Bench and LangChain behavioral evaluations, Super VL also scores above the larger Nemotron 3 Ultra in those particular tests. These are task-specific results, not proof that the smaller model is better at all reasoning problems.

Some organizations are already adapting the model for specialized work. NVIDIA says Scale AI's post-training for a vendor-credit workflow at Skydance is expected to cut inference cost by 40 times and latency by five times. Ramp expects up to 50% higher throughput and ten times lower latency for extracting structured data from receipts. AI21 and Alice report a 41% gain in an attack-resilience evaluation after customization. These outcomes are reported by NVIDIA and its partners, and some are projections rather than independently audited production results.

Why is the model relatively efficient?

The design combines several techniques. NVFP4 quantization compresses weights and activations on supported NVIDIA hardware. Multi-token prediction and the LiLiCorr speculative-decoding method aim to generate multiple token candidates with fewer expensive verification steps. Prefix and state caching reuse work when an agent repeatedly sends the same system instructions, tool definitions and conversation history.

NVIDIA also offers production serving through NIM and Dynamo. In a separately described distributed configuration using 16 B200 GPUs, its Dynamo recipe was reported to serve up to eight times as many users at a fixed per-user token rate, with lower time-to-first-token latency. That is an infrastructure optimization result, not an inherent eightfold speedup for the standalone model.

Can you run Nemotron 3.5 Super VL locally?

Yes, on suitable hardware. NVIDIA's published NVFP4 model card includes deployment options such as a DGX Spark, an H200, two H100 GPUs or a GB200, with the supported context length and concurrency depending on the configuration. Running a 120B model locally is therefore not the same as installing a small chatbot on an ordinary laptop.

Do not confuse it with the separately announced Nemotron 3.5 Bolt 74B-A9B 24GB , which NVIDIA optimized to fit in about 24 GB of memory. That Bolt model is positioned for smaller local-agent deployments and is a different release.

Where to download Nemotron 3.5 Super VL

NVIDIA has published model checkpoints on Hugging Face , alongside technical documentation, training resources and deployment instructions. The released NVFP4 checkpoint is governed by the OpenMDW 1.1 license; developers should review those terms before using it commercially. NVIDIA also lists NIM, ModelScope and hosted access through OpenRouter, with NeMo tools for domain-specific fine-tuning and reinforcement learning.

Open weights give teams more control over where and how they deploy a model, but do not make the hardware, hosting, electricity, integration or evaluation costs disappear.

Why the launch matters

The Nemotron 3.5 Super VL release is part of a broader shift in AI: enterprises increasingly need models that can reliably finish large volumes of repetitive, tool-assisted work rather than simply win one benchmark. A multimodal model that reads a scanned document, checks a chart, calls a tool and produces a structured result without switching between multiple models can make these workflows easier to operate.

For developers, the right next step is to test Nemotron on their own documents and tasks, measure end-to-end completion rates, latency and total GPU cost, and compare those results with alternative models under the same conditions. NVIDIA's fivefold figure is worth investigating; it should not be treated as a universal speed guarantee.

Sources: NVIDIA's technical breakdown , the official launch announcement and the official model card .

Source: NVIDIA Technical Blog