Artificial intelligence news.
Straight from the original announcements.

vLLM adopts LibTorch Stable ABI to ease PyTorch upgrades

November 6, 2026By the AI News Bites Editorial TeamSource: vLLM
vLLM adopts LibTorch Stable ABI to ease PyTorch upgrades
Image: vLLM

vLLM has migrated its CUDA build to PyTorch’s LibTorch Stable ABI, allowing its v0.33.0 LibTorch ABI Stable CUDA wheels to work across compatible PyTorch releases without rebuilding vLLM. The project says the change should let users upgrade sooner while reducing work for package distributors.

Key points
  • vLLM publishes LibTorch ABI Stable CUDA wheels as of v0.33.0.
  • Compatible PyTorch updates can be used without rebuilding vLLM.
  • The migration moved some meta functions and AutogradFunctions to Python.
  • The stable ABI does not remove Python, GPU or third-party compatibility constraints.

A change to how vLLM links with PyTorch

vLLM relies heavily on PyTorch and frequently calls LibTorch from its C++ backend. Previously, changes to LibTorch’s binary interface could require vLLM to be rebuilt and relinked, even when the source-level APIs remained unchanged.

The LibTorch Stable ABI provides a supported set of binary interfaces across compatible versions. vLLM says this means users can swap out the torch wheel and import it directly into an existing vLLM package. As of v0.33.0, vLLM publishes LibTorch ABI Stable CUDA wheels.

The stable interface has three layers: a C shim exposing selected LibTorch APIs, C++ wrappers that make the shim easier to use, and header-only utilities in torch::headeronly that do not link against libtorch.so.

The migration required changes across the CUDA build

vLLM began by inventorying its PyTorch API calls, identifying gaps in the stable APIs and planning replacements. Because the CUDA build contains over 175 .cpp and .cu source files, along with another 120+ headers, the project created a separate stable library and migrated kernels in manageable chunks.

The stable ABI does not currently support stable SymInt or expose PyTorch’s autograd machinery. As a result, vLLM moved its remaining meta functions and AutogradFunctions to Python. Operators also need to be registered through STABLE_TORCH_LIBRARY, with implementations wrapped using TORCH_BOX.

Many source changes were mechanical, such as replacing torch::Tensor with torch::stable::Tensor and using stable versions of existing operations. vLLM also created skills for AI agents that map existing APIs to stable equivalents and flag common migration mistakes. The project says it has not yet developed evaluations to measure those skills’ effectiveness.

Compatibility still has limits

The stable ABI covers only a subset of LibTorch. When a stable equivalent for is_pinned() was unavailable during the migration, vLLM used the general-purpose torch_call_dispatcher C shim to call the ATen operation through the dispatcher. The source says PyTorch added a stable C++ is_pinned() method in version 2.14.

The shim can add host-side overhead through type conversion, operator lookup and boxing. According to vLLM, this is usually negligible and disappears after warmup under CUDA Graphs, although it can matter in eager mode when repeatedly launching very fast kernels. The project says it saw no noticeable runtime regression during the migration.

vLLM stresses that binary compatibility is only one part of compatibility between vLLM and PyTorch. Python APIs, GPU requirements such as the minimum supported CUDA version, and third-party dependencies can still impose constraints. The project uses torch-abi-audit in continuous integration to inspect compiled libraries, identify unstable PyTorch symbols and report the minimum LibTorch version required by stable symbols.

Related news

PyTorch Helion Adds Compile-Time Heuristics to Cut Autotuning Delays
For developersNovember 12, 2026, 17:45

PyTorch Helion Adds Compile-Time Heuristics to Cut Autotuning Delays

PyTorch’s Helion now uses compile-time heuristics to select kernel settings without running slow autotuning searches. The system analyzes a kernel’s intermediate…

PyTorch team integrates SGLang with IBM Spyre in less than a week
For developersNovember 12, 2026, 15:45

PyTorch team integrates SGLang with IBM Spyre in less than a week

A two-person team integrated SGLang with IBM Spyre in less than a week, using AI coding agents to build an open source plugin of roughly 1,500 lines. PyTorch says SGLang…

vLLM adds day-0 support for Mistral Large 4, a 1-trillion-parameter MoE model
For developersOctober 29, 2026

vLLM adds day-0 support for Mistral Large 4, a 1-trillion-parameter MoE model

The vLLM inference project now supports Mistral Large 4 from day 0. According to vLLM, the model is Mistral's new frontier mixture-of-experts, natively multimodal model…