Artificial intelligence news.
Straight from the original announcements.

vLLM Adds Support for NVIDIA Vera Rubin NVL72, Cites 7.8x Throughput Gain

October 9, 2026By the AI News Bites Editorial TeamSource: vLLM
vLLM Adds Support for NVIDIA Vera Rubin NVL72, Cites 7.8x Throughput Gain
Image: vLLM

The open-source inference engine vLLM now runs on NVIDIA Vera Rubin NVL72, with daily container builds and support for models from DeepSeek, Moonshot AI, Z.ai and MiniMax. In an early look at the work, the vLLM team reports 7.8x the throughput per GPU versus GB200 NVL72 on AgentX at matched interactivity.

Key points
  • vLLM now runs on NVIDIA Vera Rubin NVL72 with daily container builds, per the vLLM project.
  • Early results: 7.8x throughput per GPU versus GB200 NVL72 on AgentX at matched interactivity.
  • Up to 3.7x higher VLM throughput versus GB300 NVL72 in MLPerf.
  • Blackwell kernels (sm100f) run on Rubin (sm107), and models from DeepSeek, Moonshot AI, Z.ai and MiniMax are supported.
  • CUDA 13.4 locality domains gave an average 1.2x speedup in a preliminary MoE decode test.

What vLLM says it has achieved so far

According to the vLLM project, engineers from Inferact, NVIDIA, Red Hat and the wider community have been bringing vLLM up on Vera Rubin NVL72 since the platform was announced. The post describes the state of the work as an early look, not a final result.

The headline numbers are 7.8x the throughput per GPU versus GB200 NVL72 on AgentX at matched interactivity, and up to 3.7x higher VLM throughput versus GB300 NVL72 in MLPerf. The team says it expects more performance as optimizations continue.

Model coverage already includes DeepSeek, Kimi, GLM and MiniMax models, according to the post.

What Rubin changes for inference

vLLM describes Vera Rubin as a platform built for agentic inference, made up of five rack-scale systems: Vera Rubin NVL72, Vera CPU rack, Groq 3 LPX, Spectrum-6 SPX and BlueField-4 STX Storage. Compared with GB200 NVL72, the post lists these per-system differences:

  • Compute: 5x more NVFP4 inference FLOPS.
  • Memory: HBM upgraded from HBM3e to HBM4, with up to 2.4x higher bandwidth.
  • Networking: sixth-generation NVLink with up to 1.7x more bandwidth, which the post says should speed up collectives such as AllReduce and All2all and help prefill/decode disaggregation and wide expert parallelism.
  • Softmax: higher exponential throughput, including 2x FP32 and 4x BF16/FP16 versus GB200, so softmax keeps pace with faster matrix operations.

Software: Blackwell compatibility and locality domains

Rubin builds on Blackwell's architecture family. It is a new GPU compile target (sm107), but kernels built for the Blackwell family target (sm100f) can also run on it. In practice, vLLM says its Blackwell kernels, especially GEMM-heavy ones such as attention and MoE, already run on Rubin without modification.

Through FlashInfer 0.7.0, vLLM also gets Rubin-tuned attention, GEMM and MoE kernels, and the team has tuned its MiniMax Sparse Attention (MSA) prefill kernel for Rubin.

A further change uses the locality domain feature in CUDA 13.4. It lets applications place computation and data in the same domain, so streaming multiprocessors (SMs) read from nearby memory with higher bandwidth and lower latency. vLLM applies this to MoE decode, which is bound by reading weights from HBM. It splits the weights column-wise across two domains and restricts each domain's SMs to their local shard.

In a preliminary test using MiniMax M3 MoE shapes, the team reports an average 1.2x speedup in small-token forward settings, with a similar trend across other TP and EP strategies. Using backfill mode engages all 212 SMs, while default mode uses 200 and still gives a similar gain. The post stresses these are early results with room for tuning, and that locality domains remain in active design and development.

What it means for users

Users can already try the setup. The post says nightly images built with CUDA 13.4 and PyTorch 2.15 are available on Docker Hub as vllm/vllm-openai:cu134-nightly for Rubin hardware. Because Blackwell kernels carry over, teams running vLLM on Blackwell should not need to start from scratch, according to the project. The performance figures are the project's own early measurements, and further optimization is expected.

Related news

Dell expands AI Factory with storage, compute and governance tools
Chips and infrastructureNovember 16, 2026, 07:00

Dell expands AI Factory with storage, compute and governance tools

Dell Technologies is expanding its Dell AI Factory with NVIDIA with persistent storage, additional compute options and supported software aimed at moving agentic AI from…

NVIDIA Says cuPhoton Correlated a 10 TB EHT Dataset in 70 Seconds
Chips and infrastructureNovember 13, 2026, 22:17

NVIDIA Says cuPhoton Correlated a 10 TB EHT Dataset in 70 Seconds

NVIDIA says its cuPhoton.xBeam software processed a 10 TB radio astronomy dataset in 70 seconds on a system with 64 GPUs. The result, according to NVIDIA, was about 460…

vLLM brings MiniMax H3-Image to Vera Rubin for subsecond 4K generation
Chips and infrastructureNovember 12, 2026

vLLM brings MiniMax H3-Image to Vera Rubin for subsecond 4K generation

vLLM says its implementation of MiniMax H3-Image reaches a median response time of 0.87 seconds for native 4096×4096 image generation on four NVIDIA Vera Rubin GPUs. A…