PyTorch Helion Adds Compile-Time Heuristics to Cut Autotuning Delays

PyTorch’s Helion now uses compile-time heuristics to select kernel settings without running slow autotuning searches. The system analyzes a kernel’s intermediate representation and dataflow, then classifies it as matmul, reduction or pointwise to choose configurations.
- Helion now selects kernel configurations at compile time.
- Kernels are classified as matmul, reduction or pointwise.
- The heuristics choose block sizes, warps, stages and other settings.
- Linear-attention tests reached 2.13x the previous no-autotune default on H100.
- vLLM reduction tests reached 2.2x the previous Helion default.
A faster alternative to per-shape tuning
Helion is PyTorch’s domain-specific language for performance portable machine learning kernels. Its performance has relied heavily on autotuning, which compiles and benchmarks candidate configurations for each kernel and shape.
PyTorch said a previous team spent 10 hours tuning a single paged-attention kernel across 72 shapes in Helion’s quick mode, and 25 at full effort. The new approach computes a configuration during compilation instead of searching for one at tuning time.
A single Helion kernel can lower into thousands of Triton kernels, with choices covering block sizes, loop orders, indexing modes and pipelining strategies. The heuristics examine device IR facts such as tiled axes, tunable block sizes and when tiles remain live in the kernel.
How Helion chooses its settings
The system routes any kernel containing a matmul operation, including hl.dot or torch.baddbmm, to the matmul heuristic. A kernel with a reduction but no matmul uses the reduction heuristic, while other kernels use the pointwise heuristic. If the kernel’s extents are not all known at compile time, Helion falls back to its original default.
The heuristics select configuration settings such as block sizes, num_warps, num_stages, L2 grouping and load eviction policies. Their resource analysis estimates register, shared memory and, on Blackwell, tensor memory use. It also estimates occupancy and register pressure to help avoid configurations that spill or launch too few cooperative thread arrays.
When autotuning is disabled, the primary heuristic supplies the default configuration. When autotuning is enabled, helper heuristics produce roughly 5, 20 related or alternative configurations to seed the search and provide promising starting points.
Reported results on H100
PyTorch reported gains across several workloads. In a test covering over 96 kernel-and-shape cells across nine linear-attention variants, the computed configuration was 1.18x faster in the forward pass than handwritten Triton in Flash Linear Attention. With backward included, it was 1.12x faster on H100. The heuristic was 2.13x faster than Helion’s previous no-autotune default on the same hardware.
Across over 153 kernel-and-shape cells from six production vLLM reduction kernels, the computed configuration was 1.27x faster than vLLM’s handwritten CUDA and C++ operators, and 2.2x faster than Helion’s previous default.
According to PyTorch, enabling the analysis changes compilation time negligibly, by tens of milliseconds, an amount it says is lost in the noise of Triton compilation. The results position the heuristics as a way for users to get strong default performance while avoiding the wait associated with broad autotuning searches.
PyTorch: Fast by Default: Compile-Time Heuristics for Helion Kernels


