Baseten says deferred tool loading cuts agent costs by 62.5%

Baseten says its deferred tool loading support for open models reduced coding benchmark costs by 62.5% and cut task times by 26.2%, without changing the model or harness. The company added the capability to DeepSeek, Kimi and GLM on its Model APIs.
- Baseten added deferred tool loading to DeepSeek, Kimi and GLM on its Model APIs.
- Coding benchmarks were 62.5% cheaper and 26.2% faster, according to Baseten.
- Agentic benchmarks were 51.1% cheaper and 14.9% faster.
- Deferred loading reduced example tool context from about 141k tokens to about 1.3k tokens.
- The tests covered 55 tasks from 8 benchmarks using GLM 5.3.
Deferred loading keeps unused tools out of the prompt
Deferred tool loading retrieves tools when an agent needs them instead of preloading every registered tool and its schema. Baseten describes the approach as progressive disclosure for tools, intended to reduce context that is carried through requests without being used.
In eager loading, the full tool list from each registered MCP or plugin is added to the system prompt on every request. Baseten says that, in its example with Slack, Notion and Baseten MCP servers, the schemas added about 141k tokens to the system prompt. Deferred loading reduced the initial tool-related context to about 1.3k tokens, consisting of tool names and a tool_search tool.
The model searches for tools when needed
Under Baseten's implementation, the model calls tool_search when it needs a tool. The client then returns tool_reference blocks for matching tools, while the server places the full schema at the bottom of the transcript rather than changing the system prompt.
According to Baseten, this preserves the key-value cache built over the shared prefix. In eager mode, each generated token attends to the full set of cached tool schemas, including tools that are never used. The company says this makes the context roughly 100 times larger in its example and increases the compute required during decoding.
Benchmarks show lower costs for coding and agentic tasks
Baseten sampled 55 tasks from 8 benchmarks and used GLM 5.3 with and without deferred tool loading. The tests ran headlessly in Claude Code. For coding benchmarks, the company registered 3 MCPs, Slack, Notion and Baseten.
Baseten reports that coding tasks were, on average, 62.5% cheaper and 26.2% faster with deferred loading. The coding group included Terminal-Bench, SWE-bench Verified, SWE-bench Pro v2 and DeepSWE v1.1. In a separate group of agentic benchmarks, costs fell by an average of 51.1% and task times by 14.9%.
The agentic group included MCP-Atlas, AutomationBench, ComplexMCP, Toolathlon-GYM and other benchmark tasks listed by the company. Baseten says deferred loading performed better than eager loading in both groups and rarely had downsides. Toolathlon-GYM, which averaged 68 tool calls, was the only benchmark where eager loading was more efficient for tool loading.
What the change means for open-model users
Baseten says deferred tool loading is generally available through its API for organizations building context-conscious internal harnesses. The company argues that support for the pattern helps open models work more like model APIs that already support deferred loading, including OpenAI and Anthropic APIs.
For users running coding or agentic systems with many MCPs, the reported results suggest that keeping unused tool schemas out of every request can reduce token-related compute and task time. The savings depend on the workload, with Baseten reporting larger benefits when setups contain many or bulky MCPs and when tasks require many requests.


