Artificial intelligence news.
Straight from the original announcements.

Cloudflare cuts AI code review cost 68% by fixing prompt caching

October 19, 2026, 21:12By the AI News Bites Editorial TeamSource: Cloudflare
Cloudflare cuts AI code review cost 68% by fixing prompt caching
Image: Cloudflare

Cloudflare says the average cost of a review by its internal AI code reviewer has fallen 68%, from $2.43 to $0.77, while code quality stays at high effort. The company also reports that the median review now takes 1m 32s instead of 4m 09s.

Key points
  • Cloudflare says its AI code review cost fell 68%, from $2.43 to $0.77 per review, at high code quality effort.
  • Median review time dropped from 4m 09s to 1m 32s.
  • Prompt cache changes lifted the cost per review from $2.47 to $2.02, as the first request now shares a prefix across repositories.
  • About 40% of savings came from code changes, the rest from model and effort choices.
  • The reviewer now runs on OpenCode v2, which agreed with v1 on approval decisions in 98% of tested merge requests.

How the bill doubled in five months

According to Cloudflare, an average review cost $1.19 when it first wrote about the tool in April. By the end of August the figure had risen to about $2.46, even though the architecture was unchanged: the same coordinator handed work to the same specialist reviewers, and the same risk tiers set the depth of each review. Around mid-September the company tasked its agents with finding where the tokens were going and cutting them.

Cloudflare's tier-adjusted chart shows the cost per review climbing from $1.22 in March to $1.44 in June, $1.74 in July and $2.32 in August. A small part came from more reviews landing in the most expensive tier, which went from 59% of reviews in March to 65% in August. The company's explanation is that more capable models lead engineers to produce bigger, more complex changes.

Most of the increase, however, came from model changes. At the start of August Cloudflare moved every reviewer onto Anthropic models, which it says performed better on this workload in its evals. That raised the cost per job from $1.38 to $2.03. The internal standards reviewer, Codex, ended up on Claude Opus 4.7, and its cost per review went from about $0.25 to $0.88, making it more expensive than the coordinator agent. Tokens per review more than tripled, from 0.91 million to 2.99 million, even as the cache hit rate rose from 85.7% in April to 97% by September.

Where the tokens were going

Before the optimization work, Cloudflare says output tokens made up about 23% of spend, cache writes 43% and cache reads 34%. Most of the money therefore went on context the model had already seen or would see again. The code quality reviewer averaged 16.6 LLM calls per review, writing 72k tokens to the cache and reading back 1.08m, because each call sends the whole conversation.

The company points to how Anthropic's prompt cache works. Only prefixes at the start of a conversation can be reused, and a prefix is stored only where a cache breakpoint ends it. Writing to the cache costs 1.25 times the normal input price, reading costs 0.1 times, and an entry expires after five minutes unless something reads it. That left two goals: make the first request of each session read from the cache, and cut the number of calls per session.

Sharing one cached prefix across repositories

From September 14 the reviewer logged the token usage of each first LLM call. The first request found a cached prefix only 14% to 26% of the time, although every session starts with the same tool definitions and agent prompt, between 11k and 21k tokens identical across all repositories.

The cause was in how OpenCode v1 built the system prompt. It put the agent prompt and an environment section (working directory and date) into one block with a single breakpoint at the end, so the shared part was never stored on its own. On September 16 Cloudflare added a marker to the end of each agent prompt and a plugin hook that splits the block in two there. The first half now has its own breakpoint. To keep it identical across jobs, the company removed anything repository-specific in front of it, disabling the list of available skills and fixing tool descriptions.

The same day it moved the Codex reviewer's reference index into the prompt, where previously every session spent its first three turns reading those files with tool calls. Codex's LLM calls per run fell from 15.9 to 9.6, and its cost per run from $1.12 to $0.72. Together the changes took the cost per review from $2.47 to $2.02.

Fewer calls, faster reviews

Further changes on September 21 and 23 took about $0.20 off every review, according to Cloudflare. Among them:

  • Removing unusable tools: sub-reviewers could see the tool that posts the final comment and tried to call it in 57% to 76% of sessions, only to be rejected. Removing it saved a call and about 2.9k tokens of tool definitions per request.
  • Parallel file reads: in 64% of the code quality reviewer's turns the model made exactly one tool call. Reviewers now read shared context and all their patches in the first response, up to 150 KB.
  • Skipping reviewers on re-reviews: about two-thirds of jobs are repeat reviews. On full-tier re-reviews, AGENTS.md reviews are skipped 56% of the time, performance reviews 23% and documentation reviews 13%.
  • Stopping after the review is posted: OpenCode used to run one more turn to write a summary nobody saw, which cost about $0.07 per job.
  • Timeouts and cancellations: a reviewer hitting its time limit is no longer treated as a provider failure, and security reviewer timeouts went from 1.8% of runs to almost none. In a sample of 446 cancelled jobs, 358 kept running, so jobs now check their status every 15 seconds.

The reviewer also moved to OpenCode v2, a ground-up rewrite. Cloudflare ran every canary-repo review on both versions in shadow mode, and they agreed on the approval decision in 98% of tested merge requests. It switched everyone in early October. With the same models, full-tier reviews on v2 finish about 13% faster.

What it means

Cloudflare says about 40% of the savings came from changes to the reviewer's code, and the rest from experimenting with which models to use and how much effort they spend. The main lesson from its account is that for agent workloads that resend long conversations, cache structure and call counts can matter as much as model price. Cloudflare says it is also testing other ways to run the reviewer, including running solely on Durable Objects with pi-durable, and promises more on that soon.

Related news

OpenAI and Databricks bring governed enterprise data to AI tools
BusinessNovember 13, 2026, 16:30

OpenAI and Databricks bring governed enterprise data to AI tools

OpenAI and Databricks are expanding their partnership to help enterprises use ChatGPT, Codex and GPT models with governed business data. The companies describe three…

Products and tools
Products and toolsNovember 12, 2026, 21:00

Amplitude says OpenAI-powered Wave ships product changes in 1, 2 days

Amplitude says its Wave product agent can help teams move from product insight to shipped improvement in 1, 2 days, down from roughly a month. The agent uses OpenAI…

Products and tools
Products and toolsNovember 12, 2026, 10:30

Holiday Extras rebuilds its app with Codex and accelerates releases

Holiday Extras says it rebuilt its main mobile app in two weeks with OpenAI Codex, replacing a development process that had previously taken more than six months. The…