Artificial intelligence news.
Straight from the original announcements.

OpenRouter finds no clear quality gap between fp8, fp4 and fp16

November 11, 2026By the AI News Bites Editorial TeamSource: OpenRouter
OpenRouter finds no clear quality gap between fp8, fp4 and fp16
Image: OpenRouter

OpenRouter found no detected accuracy difference between fp8, fp4 and fp16 in a test of Qwen3.8 27B, although the company cautions that the result covers one model and workload. Its pricing analysis found that lower precision was usually cheaper, but not in every case.

Key points
  • OpenRouter detected no accuracy difference among fp8, fp4 and fp16 across 240 scored tasks.
  • The test covered Qwen3.8 27B across six provider endpoints.
  • fp8 was cheaper than 16-bit for 14 of 24 models tested.
  • fp4 was cheaper than fp8 for 26 of 33 models tested.
  • Users can pin accepted precision with provider.quantizations.

Quantization trades memory for lower precision

LLM quantization stores a model's weights in fewer bits than the format used during training. According to OpenRouter, most current models use 16-bit FP16 or BF16, while fp8 uses 8 bits per weight and fp4 or int4 uses 4 bits.

For a 27B model, the weights require 54 GB in bf16 or fp16, 27 GB in fp8 and 13.5 GB in fp4, according to the company's table. These figures cover weights only. A running model also needs memory for its KV cache, which stores attention state for each token in the context.

Using fewer bits allows the same GPU to hold a larger model or serve more requests. The tradeoff is that each weight is rounded to a value the smaller format can store, which can reduce accuracy. OpenRouter says that post-training quantization can be dynamic or static, while quantization-aware training simulates this rounding during training.

Qwen3.8 27B results showed overlapping scores

OpenRouter tested 280 exact-answer tasks through six endpoints serving Qwen3.8 27B on October 20, 2026. Cerebras served the model at fp16, Parasail, CoreWeave and AkashML at fp8, and Darkbloom and ModelRun at fp4.

The test included multi-step arithmetic, code tracing, sorting, long-context recall and a four-step lookup chain. Across 240 scored tasks, the six endpoints scored between 98.3% and 99.2%. The fp16 Cerebras endpoint and the fp8 endpoints scored between 98.3% and 98.8%, while the fp4 endpoints scored 98.3% and 99.2%.

OpenRouter says the 95% confidence intervals on the totals overlapped, and no task family separated the precisions. In a separate 40-task multiplication test, the Darkbloom fp4 endpoint scored 26 correct answers, compared with 30 to 33 for the other endpoints. The company said the difference was too small to identify as a precision effect, with an estimated chance of about 7.5% that a gap of that size could arise without a real difference.

OpenRouter stresses that the test does not show 4-bit quantization is safe for every model or workload. It recommends testing individual prompts before moving traffic to a 4-bit endpoint.

Lower precision usually costs less, with exceptions

OpenRouter examined 1,455 provider endpoints across 470 models. Among all endpoints, 865 did not report a precision, 343 were labeled fp8, 123 fp4, 70 bf16, 20 int4, 13 nvfp4, 12 mxfp4, eight fp16 and one fp32. The company said fp8 was the most common labeled precision.

In a comparison of the cheapest endpoint at each precision, 8-bit was cheaper than 16-bit for 14 of 24 models and more expensive for seven. Four-bit was cheaper than 8-bit for 26 of 33 models and more expensive for four. The median price ratio was 0.94 for 8-bit versus 16-bit and 0.76 for 4-bit versus 8-bit.

Examples show why users cannot assume the lowest precision always has the lowest price. For Qwen3.8 27B, the listed blended price was $0.581 per million tokens at 16-bit, $0.342 at 8-bit and $0.140 at 4-bit. For gpt-oss-120b, 8-bit cost $0.100 compared with $0.067 at 16-bit. MiniMax M3 cost $0.412 at 4-bit and $0.388 at 8-bit.

For API users, OpenRouter says provider.quantizations pins the precision they receive. Listing every acceptable precision gives the router more than one endpoint to use as a fallback.

Related news

OpenRouter outlines a practical method for benchmarking LLMs
ResearchNovember 12, 2026

OpenRouter outlines a practical method for benchmarking LLMs

OpenRouter has published a guide to benchmarking large language models with application-specific tasks, code-based grading and repeatable runs. Its example study found…

OpenRouter calculates when self-hosting an LLM beats API costs
For developersNovember 12, 2026

OpenRouter calculates when self-hosting an LLM beats API costs

Self-hosting an open-weight LLM costs less than using an API only when traffic keeps the GPU busy enough, according to an OpenRouter analysis. Its calculations show that…

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context
ResearchNovember 10, 2026

Apple Study Examines Why Gated State Space Models Struggle to Learn in Context

A paper published by Apple examines how gating affects State Space Models, finding that it can steer them toward memorization in model weights and delay or prevent…