Artificial intelligence news.
Straight from the original announcements.

Modal Says NVIDIA Vera CPUs Can Double Sandbox Throughput in Early Tests

October 19, 2026By the AI News Bites Editorial TeamSource: Modal
Modal Says NVIDIA Vera CPUs Can Double Sandbox Throughput in Early Tests
Image: Modal

Modal says early benchmarks of NVIDIA's Vera CPU show up to double the task throughput of an equivalent x86 GNR machine, without slower individual tasks. The company received its first Vera server at its San Francisco office and plans to roll out Vera support for Modal Sandboxes.

Key points
  • Modal received its first NVIDIA Vera CPU server, an Arm-compatible chip, at its San Francisco office.
  • Early benchmarks show up to double the task throughput versus an equivalent x86 GNR machine, according to Modal.
  • Individual task latency was equal to or sometimes lower than on the x86 machine.
  • Modal credits Vera's high memory bandwidth for holding up under contention.
  • Supporting Arm requires changes to Modal's infrastructure and image build system; Modal plans to roll out Vera support for Sandboxes.

A first Arm server for a long-time x86 shop

According to Modal, NVIDIA hand-delivered the company's first Vera CPU server to its San Francisco office last week. Vera is NVIDIA's new Arm-compatible CPU. It was built for the Vera Rubin GPU platform but is also available as a standalone CPU server.

The move would be a big change for Modal, which has a long history of running exclusively x86 machines. The company says it would need to change its infrastructure and its image build system to support a different CPU architecture. Despite that work, Modal calls the benchmark results impressive and describes Vera as an "incredibly compelling option" for powering Modal Sandboxes.

Why Sandboxes put unusual pressure on CPUs

Modal explains that orchestrating a Sandbox product means running hundreds of concurrent containers and VMs on each cloud host. Workloads are unpredictable in several ways:

  • Resource usage can be extremely spiky. A sandbox may sit mostly idle waiting on an LLM, then surge to 10x its baseline CPU use because of an npm install or an expensive grep command.
  • Workload shapes are not known in advance, because an LLM can put anything into the sandbox.
  • Resource sizing is often a guess.

Because these mixed workloads share one pool of compute, noisy neighbor problems can arise, where one tenant's behavior affects another's performance. Modal Sandboxes support bursting, which lets users reserve guaranteed capacity for typical P50 usage instead of the worst-case P100 moment. Modal says this has saved users millions of dollars, but it increases exposure to noisy neighbor issues.

CPU time also carries wider costs, the company argues. For background agents, such as Ramp Inspect or those running on Niteshift, faster agents mean faster answers and lower CPU time. For reinforcement learning, GPUs sit idle while sandboxed processes run, and most training setups are still synchronous, so any time saved on CPU is time saved on GPU.

The benchmark: Agentless Agent Bench

To test Vera, Modal built Agentless Agent Bench, a collection of 50 Terminal-Bench oracle solutions and verifiers. They are stripped of all network calls and prewarmed in concurrent Docker containers, so that only the work done on the CPU remains, such as compilation, file grepping, git operations and device emulation.

In the first test, run serially on a machine without load, Modal's author says he was skeptical. Across the 50 tasks, Vera often performed better than the other CPUs, but not always, and not by a large margin.

Under load, the picture changed. The throughput test raises the number of concurrent containers and loops as many task completions as possible in a fixed time. Modal reports up to double the task throughput on Vera compared with an equivalent x86 GNR machine, while individual task latency was equal to, or sometimes lower than, the x86 machine. According to Modal, Vera does not slow down as jobs accumulate. The company says the Vera team confirmed the results were not surprising.

What it could mean for Sandbox users

Modal attributes Vera's performance under contention to its high memory bandwidth, which it says allows significantly more container workloads per core without CPU throttling. As CPU counts rose, the alternatives hit early throughput plateaus while Vera kept improving. Modal suggests it could push Vera further than the node it had for benchmarking.

For Modal, this means greater packing density and more forgiving burst protection. For Sandbox users, the company says it can offer higher absolute scale and throughput without sacrificing latency. Modal says it will roll out support for Vera CPUs for Sandboxes and work with NVIDIA and cloud providers to scale up its fleet. The results are early and come from Modal's own benchmark, and the source gives no timeline or pricing.

Related news

Dell expands AI Factory with storage, compute and governance tools
Chips and infrastructureNovember 16, 2026, 07:00

Dell expands AI Factory with storage, compute and governance tools

Dell Technologies is expanding its Dell AI Factory with NVIDIA with persistent storage, additional compute options and supported software aimed at moving agentic AI from…

Vultr Named a Major Player in IDC Public Cloud AI Assessment
Chips and infrastructureNovember 12, 2026, 16:21

Vultr Named a Major Player in IDC Public Cloud AI Assessment

Vultr has been named a Major Player in the IDC MarketScape: Worldwide Public Cloud AI Infrastructure as a Service 2026-2027 Vendor Assessment. The company says the…

NVIDIA Gives OpenShell and AI Infrastructure Tools to Open Source
For developersNovember 10, 2026, 17:00

NVIDIA Gives OpenShell and AI Infrastructure Tools to Open Source

NVIDIA is contributing its OpenShell runtime for securing autonomous agents to the Cloud Native Computing Foundation, alongside work on Agent Substrate and new AI…