AI inference capacity grew 4.6× in one year, Epoch AI finds

The estimated agentic inference capacity of AI chips sold to date grew 4.6× between Q2 2025 and Q2 2026, according to Epoch AI. That was faster than the 2.6× increase in the chips’ peak 8-bit computing power.
- Agentic inference capacity grew 4.6× from Q2 2025 to Q2 2026.
- Peak 8-bit computing power grew 2.6× over the same period.
- Blackwell and Blackwell Ultra supplied 74% of added capacity.
- Inference capacity per unit of peak compute rose about 77%.
- Epoch AI expects the growth rate to slow as the installed base expands.
Blackwell drove most of the increase
Epoch AI attributes the growth largely to Nvidia’s transition from Hopper to Blackwell. Nvidia’s Blackwell and Blackwell Ultra families accounted for 74% of the added agentic inference capacity during the period, according to the research.
By Q2 2026, those two families represented roughly 73% of total agentic inference capacity, compared with 50% of total peak 8-bit computing power. The report says GB300 systems deliver roughly 8× the inference capacity of H200s per GPU.
The estimate measures cumulative capacity from chips sold to date, rather than capacity added in a single quarter. The total reached 4.58 times its Q2 2025 level in the data, while peak 8-bit compute reached 2.59 times its starting level.
How Epoch AI measured agentic capacity
Epoch AI used version 1 of its Inference Performance Index, or IPI, to compare hardware on a fixed set of workloads with long, cached contexts. The index is expressed in GB300 equivalents and is applied to estimates from Epoch AI’s AI Chip Sales dataset.
The measure accounts for memory, inter-chip communication and lower-precision computation. It includes both input processing and output generation, and uses the same workload basket for every quarter.
To estimate total capacity, Epoch AI multiplied the cumulative number of chips sold in each category by its per-chip IPI score, then summed the results. It calculated peak 8-bit computing power for the same chips as a comparison.
Epoch AI estimates that inference capacity per unit of peak compute rose by about 77% over the year. The organization uses “hardware, model co-design” broadly for gains beyond proportional increases in arithmetic throughput and memory bandwidth, including larger scale-up domains, lower-precision computation and model architectures that use hardware more effectively.
Growth is expected to slow, but the gap may persist
Chips sold in Q2 2026 added 3.3× the inference capacity of chips sold in Q2 2025. Because that addition was smaller than the 4.6× growth in cumulative capacity, Epoch AI expects the pace of total growth to slow as the installed base becomes larger.
The report says sustaining 4.6× annual growth would require yearly additions equal to 3.6× the capacity at the beginning of each year. It also expects smaller improvements in later chip generations. Modeled inference capacity per GPU rises 8× from H200 to GB300, but 2.2× from GB300 to Vera Rubin VR200.
Epoch AI says continued hardware, model co-design should nevertheless keep inference capacity growing faster than peak computing power. Its uncertainty analysis gives 90% intervals of 4.2, 5.1× for inference growth and 2.5, 2.7× for peak compute growth, covering shipment and hardware-mix uncertainty.
Epoch AI: AI inference capacity grew 4.6× in one year


