Artificial intelligence news.
Straight from the original announcements.

Apple Study: Elaborate Harnesses Add Little for Strong ML Engineering Agents

October 1, 2026By the AI News Bites Editorial TeamSource: Apple
Apple Study: Elaborate Harnesses Add Little for Strong ML Engineering Agents
Image: Apple

A new paper published on Apple's Machine Learning Research site finds that, under an equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses offer no advantage over a single session of a minimal-harness coding agent. The authors conclude that the backbone model is the primary driver of performance in autonomous machine learning engineering.

Key points
  • Under equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses gave no advantage over a single session of a minimal-harness coding agent.
  • The minimal agent gets direct access to the execution environment through read, write and bash primitives.
  • Ablation studies led the authors to argue that the machinery layers become redundant in the coding agent setting.
  • The authors conclude that hand-crafting harnesses around strong models yields poor returns for current MLE benchmarks.
  • The paper is by Kirill Brilliantov, Alejandro Hernández-Cano and Emmanuel Abbé, with EPFL affiliations.

What the researchers tested

The paper, titled "How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?", is by Kirill Brilliantov, Alejandro Hernández-Cano and Emmanuel Abbé. Brilliantov and Hernández-Cano contributed equally, and the affiliations listed are EPFL, with a note that Hernández-Cano's work was done while at Apple.

The study looks at autonomous machine learning engineering (MLE) agents, which according to the authors have made significant progress on public leaderboards. Many of these agents, they write, are built on increasingly elaborate machinery, such as multi-agent orchestrators and dedicated retrieval subagents. The authors say this is often motivated by progress stagnating over long-horizon cycles and by the limited primitives available to large language models.

A minimal baseline against elaborate harnesses

The researchers point to a gap in the field. While harnesses keep expanding, they say, simpler but improved coding agents have received little attention. In this setup, the LLM has direct access to the execution environment through read, write and bash primitives.

Using such a minimal-harness coding agent as a baseline, the authors compared it with open-source state-of-the-art harnesses. Both used the same frontier LLM backbone and an equal time budget. The baseline ran as a single session. According to the paper, the elaborate harnesses provided no advantages over it.

The authors say this points to the backbone model as the primary driver for performance.

Ablations and the authors' conclusion

Beyond the head-to-head comparison, the paper reports a series of large-scale systematic ablation studies. From these, the authors argue that the machinery layers become redundant in the coding agent setting.

Their conclusion is direct: the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE benchmarks. The qualifier matters, since the claim is tied to current MLE benchmarks and to strong frontier models, not to every agent task or every model.

What it means for agent builders

For teams building autonomous ML engineering agents, the findings suggest that investment in multi-agent orchestration and retrieval subagents may deliver less than expected when a strong model already has direct access to the execution environment. The authors' results point toward choosing a capable backbone and keeping the surrounding system simple, at least for the benchmarks they studied.

The paper is classified under Methods and Algorithms research on Apple's site. The source text does not give specific benchmark scores, model names or datasets, so the size of any differences between the setups is not stated here.

Related news

GPT-6 Astra led coding agents, but finished only 26.4% of tasks
ResearchNovember 10, 2026, 17:08

GPT-6 Astra led coding agents, but finished only 26.4% of tasks

OpenAI’s GPT-6 Astra passed an average of 69.9% of hidden acceptance criteria in a benchmark of coding agents, yet fully finished only 26.4% of tasks. The Co-Design…

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context
ResearchNovember 10, 2026

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context

A paper published by Apple examines how gating affects State Space Models, finding that it can steer them toward memorization in model weights and delay or prevent…

Apple Study Questions Whether LLM Judges Reflect Human Writing Preferences
ResearchNovember 10, 2026

Apple Study Questions Whether LLM Judges Reflect Human Writing Preferences

Apple researchers have examined how reliably large language models can judge whether generated writing matches a user’s preferred style. The study finds that judge…