Artificial intelligence news.
Straight from the original announcements.

Baseten says deferred tool loading cuts agent costs by 62.5%

November 19, 2026, 18:49By the AI News Bites Editorial TeamSource: Baseten
Baseten says deferred tool loading cuts agent costs by 62.5%
Image: Baseten

Baseten says its deferred tool loading support for open models reduced coding benchmark costs by 62.5% and cut task times by 26.2%, without changing the model or harness. The company added the capability to DeepSeek, Kimi and GLM on its Model APIs.

Key points
  • Baseten added deferred tool loading to DeepSeek, Kimi and GLM on its Model APIs.
  • Coding benchmarks were 62.5% cheaper and 26.2% faster, according to Baseten.
  • Agentic benchmarks were 51.1% cheaper and 14.9% faster.
  • Deferred loading reduced example tool context from about 141k tokens to about 1.3k tokens.
  • The tests covered 55 tasks from 8 benchmarks using GLM 5.3.

Deferred loading keeps unused tools out of the prompt

Deferred tool loading retrieves tools when an agent needs them instead of preloading every registered tool and its schema. Baseten describes the approach as progressive disclosure for tools, intended to reduce context that is carried through requests without being used.

In eager loading, the full tool list from each registered MCP or plugin is added to the system prompt on every request. Baseten says that, in its example with Slack, Notion and Baseten MCP servers, the schemas added about 141k tokens to the system prompt. Deferred loading reduced the initial tool-related context to about 1.3k tokens, consisting of tool names and a tool_search tool.

The model searches for tools when needed

Under Baseten's implementation, the model calls tool_search when it needs a tool. The client then returns tool_reference blocks for matching tools, while the server places the full schema at the bottom of the transcript rather than changing the system prompt.

According to Baseten, this preserves the key-value cache built over the shared prefix. In eager mode, each generated token attends to the full set of cached tool schemas, including tools that are never used. The company says this makes the context roughly 100 times larger in its example and increases the compute required during decoding.

Benchmarks show lower costs for coding and agentic tasks

Baseten sampled 55 tasks from 8 benchmarks and used GLM 5.3 with and without deferred tool loading. The tests ran headlessly in Claude Code. For coding benchmarks, the company registered 3 MCPs, Slack, Notion and Baseten.

Baseten reports that coding tasks were, on average, 62.5% cheaper and 26.2% faster with deferred loading. The coding group included Terminal-Bench, SWE-bench Verified, SWE-bench Pro v2 and DeepSWE v1.1. In a separate group of agentic benchmarks, costs fell by an average of 51.1% and task times by 14.9%.

The agentic group included MCP-Atlas, AutomationBench, ComplexMCP, Toolathlon-GYM and other benchmark tasks listed by the company. Baseten says deferred loading performed better than eager loading in both groups and rarely had downsides. Toolathlon-GYM, which averaged 68 tool calls, was the only benchmark where eager loading was more efficient for tool loading.

What the change means for open-model users

Baseten says deferred tool loading is generally available through its API for organizations building context-conscious internal harnesses. The company argues that support for the pattern helps open models work more like model APIs that already support deferred loading, including OpenAI and Anthropic APIs.

For users running coding or agentic systems with many MCPs, the reported results suggest that keeping unused tool schemas out of every request can reduce token-related compute and task time. The savings depend on the workload, with Baseten reporting larger benefits when setups contain many or bulky MCPs and when tasks require many requests.

Related news

GitLab Connects Coding Agents to Automated Review, CI and Security Workflows
Products and toolsNovember 19, 2026

GitLab Connects Coding Agents to Automated Review, CI and Security Workflows

GitLab is introducing a configuration that connects coding agents including Claude Code, Codex, Cursor, Devin and OpenCode with GitLab Duo workflows for code review…

LangChain adds cross-agent tracking to help control coding costs
For developersNovember 18, 2026, 20:33

LangChain adds cross-agent tracking to help control coding costs

LangChain is expanding LangSmith to give teams a shared view of spending and activity across multiple coding agents. The company says the tools can help organizations…

LangSmith links coding agent spending to pull request outcomes
Products and toolsNovember 18, 2026, 16:00

LangSmith links coding agent spending to pull request outcomes

LangChain has added coding agent reporting to LangSmith, linking spending to GitHub pull requests and GitLab merge requests. The company says the feature is designed to…