Artificial intelligence news.
Straight from the original announcements.

Google makes Pathways generally available for large-model training

November 9, 2026By the AI News Bites Editorial TeamSource: Google for Developers
Google makes Pathways generally available for large-model training
Image: Google for Developers

Google has announced the general availability of Pathways on Google Cloud Platform, a distributed system it built to train large models including Gemini. Pathways uses a single Python client to describe an entire computation, while its runtime coordinates placement, data transfers and execution across accelerators.

Key points
  • Pathways is generally available on Google Cloud Platform.
  • Google built the system to train large models including Gemini.
  • A single Python client describes the full computation as a graph.
  • The runtime coordinates placement, transfers, dependencies and ordering.
  • Pathways runs JAX programs on TPUs.

A response to the limits of SPMD training

Most large-model training uses the Single Program, Multiple Data, or SPMD, model. In this approach, every accelerator runs the same program in lockstep and coordinates with collective operations such as AllReduce. Google says this works well when a job is uniform, but becomes harder to manage when different parts of a computation need to run at the same time.

The company points to pipeline parallelism and reinforcement learning post-training as examples. In a pipeline, devices run different stages and can spend time waiting for activations or gradients. In an RL loop, a sampler, reward model and trainer are separate programs that need to overlap. SPMD makes these dependencies harder to coordinate because each process has visibility into only its local devices and data.

One client describes the whole computation

Pathways takes a different approach by representing an end-to-end computation as a graph. The blocks of work form nodes, tensors moving between them form edges, and devices provide the locations where the nodes run.

With Pathways, developers define that computation from a single Python client without writing custom coordination code. The client runs on an ordinary CPU machine and works with virtual devices rather than directly managing physical chips. According to Google, a resource manager maps those virtual devices onto real accelerators, while workers on accelerator hosts run the compiled programs they receive.

This design separates the description of the computation from the hardware on which it runs. Google says the runtime handles transfers, dependencies and ordering along the graph, while the virtual-device model allows the devices used for a computation to be changed.

Designed for large, changing workloads

Google says large training runs also need to continue through hardware failures. During Gemini 2.5 training, hardware failures interrupted runs multiple times per hour, according to the company. In an SPMD setup, one lost machine can prevent a collective from completing, stop the job and force a restart from the last checkpoint.

The company also describes limits around resource ownership. In the SPMD model, processes keep their accelerators for the duration of a run, even when one stage needs fewer chips than another. Devices cannot be added or returned during execution, and separate jobs require separate allocations and processes.

Pathways is now available on Google Cloud and runs JAX programs on TPUs. For teams training large models, the announcement offers Google's own coordination system as a way to describe more complex workloads centrally instead of managing every process and data dependency separately.

Related news

PyTorch team integrates SGLang with IBM Spyre in less than a week
For developersNovember 12, 2026, 15:45

PyTorch team integrates SGLang with IBM Spyre in less than a week

A two-person team integrated SGLang with IBM Spyre in less than a week, using AI coding agents to build an open source plugin of roughly 1,500 lines. PyTorch says SGLang…

PyTorch and TorchTitan Train a 1T-Parameter MoE on JUPITER
ResearchNovember 12, 2026, 13:00

PyTorch and TorchTitan Train a 1T-Parameter MoE on JUPITER

The EUROPA consortium says it has scaled pretraining for a roughly 1-trillion-parameter Mixture-of-Experts model on JUPITER, a publicly funded European supercomputer.…

PyTorch Models Can Now Run on GPUs Inside Web Browsers
For developersNovember 10, 2026, 17:45

PyTorch Models Can Now Run on GPUs Inside Web Browsers

PyTorch models can now run on a device’s GPU inside the web browser through the new ExecuTorch WebGPU backend. The approach uses a single ExecuTorch .pte file, avoids…