Artificial intelligence news.
Straight from the original announcements.

Qodo Benchmark: Claude Opus 5.5 Leads Code Review, Open-Weight GLM 5.3 Close Behind

October 12, 2026, 12:20By the AI News Bites Editorial TeamSource: Qodo
Qodo Benchmark: Claude Opus 5.5 Leads Code Review, Open-Weight GLM 5.3 Close Behind
Image: Qodo

Claude Opus 5.5 at high reasoning scored the best F1 (56.8%) in Qodo's latest code review benchmark, while open-weight GLM 5.3 placed fourth and finished ahead of every OpenAI and Google configuration tested. Qodo, which sells a code review and governance platform, tested 13 models in 38 model and reasoning-level configurations.

Key points
  • Qodo tested 13 models in 38 configurations on 125 pull requests containing 703 known, real bugs.
  • Claude Opus 5.5 at high reasoning led with 56.8% F1, followed by Claude Sonnet 5.5, Claude Fable 5.1 and GLM 5.3 (52.1%).
  • Open-weight GLM 5.3 scored ahead of every OpenAI and Google configuration tested, at $0.76 per PR.
  • Higher reasoning usually raises recall and lowers precision, and the best level differs by model.
  • Cost per PR ranges from about two cents (GPT-6 Luna at xhigh) to $3.63 (Gemini 3.8 Flash at high).

How Qodo built the test

According to Qodo, most code review benchmarks rely on synthetic bugs or hand-labeled issues. Its own benchmark starts from bugs that actually happened. The company takes bugs that were later found and fixed, traces each one to the pull request that introduced it, and asks the reviewer to catch it in that original PR. The dataset covers 125 pull requests containing 703 known bugs.

To isolate the model, Qodo holds everything else fixed: a single reviewer, standard review effort, and the explorer subagent turned off. Findings are scored against the known bugs by an LLM judge, Claude Sonnet 4.6. Qodo reports precision, recall, F1 (its headline quality score), recall on backtracked bugs, cost per PR and median review time.

The ranking

Four models sit within 4.7 F1 points of each other: Claude Opus 5.5 (56.8%), Claude Sonnet 5.5, Claude Fable 5.1 (53.5%) and GLM 5.3 (52.1%). A second group between 46% and 50% includes DeepSeek V4 Pro, Kimi K3, GPT-6 Sol, Gemini 3.8 Flash and GPT-6 Astra. At the bottom of the table are Claude Haiku 4.5 (30.4%), Laguna S 2.1 (27.3%) and Mistral Medium 3.5 (24.0%).

Qodo says rankings can reorder from one month to the next, which is why it reruns the benchmark with every major release. Results reflect runs completed between September and October 2026, and some configurations were run once while others were averaged across repeated passes.

Reasoning level and cost matter as much as the model

Raising the reasoning level usually increases recall and lowers precision, according to Qodo, and the net effect on F1 depends on the model.

  • Claude Opus 5.5 scores 56.8% F1 at high and 54.0% at xhigh. At xhigh it catches the most backtracked bugs of any configuration (55.7%), but precision drops to 54.4%.
  • GPT-6 Sol scores 48.7% at medium and 48.4% at xhigh. Xhigh costs almost three times as much ($0.51 vs. $0.18 per PR) and takes more than twice as long (224s vs. 101s).
  • Claude Fable 5.1 scores 53.4% at low and 53.5% at high, at a similar cost per PR.

Cost varies by more than 100x, Qodo says. GPT-6 Luna at xhigh reaches 41.7% F1 for about two cents per PR. Gemini 3.8 Flash at high reaches 47.0% F1 for $3.63 per PR, with a median review time of almost 14 minutes. Claude Sonnet 5.5 at high reaches 51.7% F1 for $0.31 per PR in 90 seconds, which Qodo says is close to the top tier at a third of the cost of Claude Opus 5.5. At xhigh, Sonnet scores 54.9% F1 at $0.73 per PR.

Open weights and different failure modes

GLM 5.3 reaches 52.1% F1 at $0.76 per PR. DeepSeek V4 Pro (49.9%) and Kimi K3 (49.0%) also finish ahead of GPT-6 Astra and Gemini 3.8 Flash. For teams that need open weights for data residency, cost or control, Qodo says the quality gap to the frontier is now a few points of F1.

Models with similar F1 can behave very differently. Gemini 3.8 Flash at high has the highest precision in the table (71.0%) but catches only 35.1% of known bugs. GPT-6 Sol at xhigh is more balanced, with 48.4% precision and 49.3% recall. Qodo argues that a team wanting few, high-confidence comments and a team wanting maximum coverage should not run the same configuration.

What it means for teams

Qodo says the best-performing combination of model, reasoning level and harness configuration is what runs in its product, and production changes when a new release shifts the ranking. The company says it is not tied to one provider, runs a mix by default, and can switch to a specific provider or open-weight model for compliance, data residency or contract reasons. It also says it can select a model and reasoning level for each review.

For organizations with approved LLM providers, Qodo positions the benchmark as an index for code review, showing what a current stack delivers and where another model or reasoning level could catch more bugs or cut cost. The company notes that the benchmark uses its own internal pull requests, that cost excludes evaluator cost and failed attempts, and that the best choice depends on whether a team prioritizes coverage, precision, cost or speed.

Related news

JetBrains asks 12 AI models for 1,200 programming hot takes
ResearchNovember 15, 2026, 18:02

JetBrains asks 12 AI models for 1,200 programming hot takes

Twelve AI models overwhelmingly attacked Python, Rust and TypeScript in a JetBrains experiment that collected 1,200 opinions about programming languages. The models also…

GPT-6 Astra led coding agents, but finished only 26.4% of tasks
ResearchNovember 10, 2026, 17:08

GPT-6 Astra led coding agents, but finished only 26.4% of tasks

OpenAI’s GPT-6 Astra passed an average of 69.9% of hidden acceptance criteria in a benchmark of coding agents, yet fully finished only 26.4% of tasks. The Co-Design…

Claude Fable 5.2 arrives on Vercel’s AI Gateway
ModelsNovember 9, 2026

Claude Fable 5.2 arrives on Vercel’s AI Gateway

Anthropic’s Claude Fable 5.2 is now available through Vercel’s AI Gateway, giving developers access to a model designed for open-ended technical work. The model supports…