Artificial intelligence news.
Straight from the original announcements.

Apple's RLTL;DR Lets AI Learn From Its Own Failures on Tasks It Cannot Solve

October 1, 2026By the AI News Bites Editorial TeamSource: Apple
Apple's RLTL;DR Lets AI Learn From Its Own Failures on Tasks It Cannot Solve
Image: Apple

Apple researchers have introduced RLTL;DR, a training method that lets a model improve on tasks it almost never solves by writing short feedback on its own failed attempts. According to the paper, a Qwen 3.5 9B Thinking policy that stayed flat at a Pass@1 of 0% to 1% under standard GRPO training reached 14, 31% with the new approach.

Key points
  • Apple's RLTL;DR has a model write a single TL;DR insight after each failed attempt and condition later rollouts on those insights.
  • On tool-calling and coding tasks filtered to Pass@128 = 0, standard GRPO on Qwen 3.5 9B Thinking stays at 0% to 1% Pass@1.
  • RLTL;DR reaches 14, 31% Pass@1 with insights in context, and 12, 13% with no insight in context at eval time.
  • The SFTL;DR variant, trained on only 4k (task, insight) tuples, recovers almost the full performance of RLTL;DR and classical SFT on full rollouts.

The problem with learning from rare successes

The paper, published in October 2026 and tied to NeurIPS, starts from a limitation of reinforcement learning with verifiable rewards (RLVR). In this common setup, an agent makes multiple attempts at a task and the training optimizes toward the successful ones.

According to the authors, this becomes a problem in self-improvement settings. There, tasks can be so difficult that the agent has a low or even no chance of success. There are also no teacher models or example solutions to distill from. If almost every attempt fails, there is little signal to learn from.

How RLTL;DR works

In RLTL;DR, the policy is shown the verifier outputs after each failed attempt and then writes its own feedback in the form of a single TL;DR insight. The next rollout is conditioned on all previous insights, and rollouts are sampled sequentially until a solution is found.

The authors add a second step: they enable backpropagation on the in-context insights. The goal is to internalize a direct task → insight mapping, so that the model carries the lesson without needing it spelled out in the prompt.

The paper is by Michael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi, Abbas Kazerouni, Omar Attia, Sanjoy Chowdhury and Alexander Toshev.

Results on hard tool-calling and coding tasks

The evaluation used challenging tool-calling and coding datasets, filtered to Pass@128 = 0. On these, standard GRPO training of a Qwen 3.5 9B Thinking policy stays flat at a Pass@1 of 0% to 1%, according to the paper.

RLTL;DR, the authors report, breaks through this learning barrier. It achieves a Pass@1 of 14, 31% with insights in context during training. The authors stress that it also reaches 12, 13% when no insight is in context at evaluation time.

They identify the task → insight internalization as the key ingredient.

SFTL;DR: a compact training variant

To study this further, the team reduced the approach to SFTL;DR. It trains only on (task, insight) tuples, without showing or backpropagating on any rollouts. According to the paper, training on only 4k of these tuples recovers almost the full performance of RLTL;DR and of classical SFT on full rollouts.

The authors say this points to a promising compacted training paradigm of the form "on this sort of task, keep this sort of thing in mind", and they hope it inspires future research.

What it means

The results suggest that when a model has no teacher and almost never succeeds, its own condensed feedback on failures can still provide a usable learning signal. The reported gains are measured on a specific model and on datasets filtered to be very hard, so the paper does not by itself show how the method performs elsewhere. The finding that short insights can stand in for far larger sets of full rollouts is the part the authors single out for follow-up work.

Related news

Apple Paper Defines Theoretical Limits for Language Model Alignment
ResearchNovember 12, 2026

Apple Paper Defines Theoretical Limits for Language Model Alignment

Apple researchers have published a NeurIPS paper defining the maximum reward improvement language models can achieve under a fixed KL-divergence budget. The work also…

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context
ResearchNovember 10, 2026

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context

A paper published by Apple examines how gating affects State Space Models, finding that it can steer them toward memorization in model weights and delay or prevent…

Apple Study Questions Whether LLM Judges Reflect Human Writing Preferences
ResearchNovember 10, 2026

Apple Study Questions Whether LLM Judges Reflect Human Writing Preferences

Apple researchers have examined how reliably large language models can judge whether generated writing matches a user’s preferred style. The study finds that judge…