Forethought Defines Compute Extrapolation in Test-Time Scaling

A new Forethought essay distinguishes two forms of test-time scaling, arguing that some AI systems can keep improving when given far more per-task compute than they used during training. James Tillman calls this capability “compute extrapolation” and says it could affect AI progress, evaluations and race dynamics.
- Forethought distinguishes ordinary test-time scaling from compute extrapolation.
- Compute extrapolation allows performance to keep improving beyond typical training-time compute.
- The essay proposes measuring extrapolation through a ceiling and a slope.
- Tillman says it could affect AI progress, evaluations and race dynamics.
Two ways to scale compute at test time
Test-time scaling trades additional computation for better performance on a task. The essay says one form is limited by the maximum amount of per-task compute used during training. In this case, giving a system more compute can improve results only up to that training-time limit.
The second form continues to produce performance gains beyond that limit. Tillman calls this compute extrapolation. The concept can apply to several learned scaling methods, including the length of chain-of-thought for an LLM trained with RLVR, the number of loops in a looped Transformer, and the number of agents in a multi-agent LLM swarm.
As an example, the essay considers an LLM trained with a maximum of 100k chain-of-thought tokens. Under ordinary test-time scaling, more tokens would help only up to 100k. With compute extrapolation, one million or one billion tokens could continue to improve performance.
A definition based on typical training workloads
Tillman argues that compute extrapolation should not be defined by rare outliers in a training run. Instead, he defines it relative to the per-task compute at the 99th percentile of total compute usage. If 99% of total training compute is spent on tasks using at most N FLOPs per task, then continued improvement at 10x N or 1000x N at test time would indicate some degree of compute extrapolation.
The essay says the specific 99% threshold is less important than ensuring that there is negligible training compute above the selected task size, while most learning occurs below it. This approach is intended to make measurements less sensitive to individual tasks that receive unusually large amounts of compute.
Tillman proposes two ways to characterize the capability. The ceiling is the multiple of N at which performance stops rising. The slope measures how closely performance beyond N follows a function fitted to the empirical relationship between FLOPs and performance below N. Both measures could vary by domain.
Why compute extrapolation could matter
According to the essay, compute extrapolation likely accelerates progress in the most impressive AI capabilities, makes dangerous capability evaluations harder and changes AI race dynamics. Tillman writes that it appears to be more recent than test-time scaling in general, and that AI labs are still mastering it, perhaps only in some domains.
The essay says its ceiling and slope could improve through better LLM context management, multi-agent scaling and other advances. It may be easier to achieve high ceilings in areas such as math or cyber, where incremental progress is easier to recognize, than in domains where quality improvements are more difficult to identify.
Tillman also notes that compute extrapolation is broader than LLM-specific methods. He gives AlphaGo Zero as an example of a non-general, non-LLM AI trained with 1,600 simulations per move that could use more simulations at test time while continuing to improve. The distinction gives researchers a way to examine whether test-time compute remains useful beyond the workloads represented during training.
Forethought: Two Kinds of Test-Time Scaling


