Apple Paper: Language Model Agents Can Learn When to Compact Their Own Context

A new paper published on Apple's Machine Learning Research site proposes SELFCOMPACT, a scaffold that lets a language model decide for itself when and how to compact its accumulated context. According to the authors, it matches or exceeds fixed-interval summarization at a fraction of the token cost, without any fine-tuning.
- SELFCOMPACT lets a language model decide when and how to compact its own context.
- It pairs a compaction tool with a lightweight rubric on when to fire and when to suppress, with no fine-tuning.
- Tested on six benchmarks and seven models in competitive math and agentic search.
- Up to 18.1 points better on math and 5, 9 points on agentic search than a no-summarization baseline, at 30, 70% lower per-question cost.
- The paper is listed under NeurIPS, with most authors at Johns Hopkins University.
The problem with long agent traces
The paper, titled "Self-Compacting Language Model Agents" and listed under the NeurIPS conference, starts from a familiar issue for AI agents. Long traces made of chains of thought and tool calls pile up stale content. According to the authors, this content anchors later generations and eventually outgrows the context window.
Existing scaffolds deal with the issue through fixed-interval compaction, which is triggered when a token threshold is reached. The authors argue that such triggers ignore the structure of the trajectory. As a result, they risk discarding partial results in the middle of a derivation or a search.
How SELFCOMPACT works
SELFCOMPACT hands the timing decision to the model. It combines two elements that work at inference time:
- A compaction tool that the model can invoke to summarize the context it has accumulated.
- A lightweight rubric that says when to fire (a sub-task has resolved, or the trajectory is converging) and when to suppress (mid-derivation, or when stuck).
The authors say both parts are needed. The tool alone is used unevenly across open-weight models, which often call it at unhelpful moments or not at all. The rubric alone cannot act. Together, they produce effective adaptive compaction "without any fine-tuning or external supervision", according to the paper.
Reported results
The team reports empirical results on six benchmarks, covering competitive math and agentic search, and on seven models. According to the paper, SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost.
Compared with a no-summarization baseline, the authors report improvements of up to 18.1 points on math and 5, 9 points on agentic search. Per-question cost is 30, 70% lower, according to the paper.
The authors describe a "meta-cognitive gap": unprompted models cannot reliably tell when their own context is rotting. A lightweight rubric closes that gap, in their words, reframing when to compact as a capability that scaffolds can supply without training.
What it means
For teams building agents on open-weight models, the paper points to a way of managing long-running tasks that requires no additional training. Based on the reported results, the approach could cut token spending while keeping or improving accuracy on math and search tasks. The findings come from the authors' own experiments on the listed benchmarks and models, and the source does not say whether the method is used in any Apple product.
Most of the authors are affiliated with Johns Hopkins University, according to the paper's listing.
Apple: Self-Compacting Language Model Agents


