Together AI launches free storage caching for GPU clusters

Together AI has launched Together Storage Accelerator, a distributed caching layer for its GPU Clusters that serves repeated data reads from local NVMe storage. The company says the service is available now at no additional cost and can accelerate access to object storage and Hugging Face Hub data.
- TSA is available now at no additional cost on Together GPU Clusters.
- It caches objects on cluster NVMe after the first read.
- Together AI measured about 20 times faster warm reads for a 2 GiB object.
- A 200-node test reduced origin fetches from 200 to one.
- Existing storage and Hugging Face workflows can use TSA without application changes.
Caching data closer to the GPUs
Together Storage Accelerator, or TSA, runs on every GPU node in a cluster and provides endpoints for existing storage workflows. When a node reads an object for the first time, TSA stores it on cluster NVMe. Later reads from any node can then be served from inside the cluster rather than fetching the object again from a remote origin.
Together AI says this addresses a common bottleneck in AI workloads. Teams may keep source data in Amazon S3, Cloudflare R2 or Together AI's T4 object storage, or download models from Hugging Face Hub. Cross-cloud and cross-region transfers can add latency, data egress costs and rate limits, particularly when many nodes request the same checkpoint.
Tests show faster warm reads
In measurements from October 28, 2026, on an H100 node in a Together GPU Cluster in us-central, a 2 GiB object from a Cloudflare R2 bucket was read at 143 to 169 MB/s without TSA, taking 12.7 to 15.0 seconds. A warm TSA read reached 3.0 to 3.4 GB/s and took 0.64 to 0.72 seconds, which Together AI reports as about 20 times faster.
For small files from the same bucket, warm TSA reduced read latency from 320 milliseconds to 5 milliseconds for a 5 KB file, and from 680 milliseconds to 8 milliseconds for a 2.4 MB file. In a 200-node test pulling the same checkpoint, TSA reduced origin fetches from 200 to one.
The company says the first read through TSA ran within about 5% of a direct download. It also cautions that results depend on the origin, object sizes and network path. The measurements used single-stream reads unless otherwise noted and were verified against a direct download with MD5.
No application rewrite required
Together AI says users can enable TSA without changing their applications, adding a new SDK or rewriting data loaders. A single endpoint configuration supports S3-compatible storage, including S3, R2 and T4, while another endpoint can be used for Hugging Face Hub. Existing boto3 code, transformers calls and command-line workflows continue to work as-is.
Credentials are configured by the user and handled by TSA through an encrypted secrets vault. The technology is built on ASTRA, which Together AI describes as its Accelerated Storage Transport & Retrieval Architecture. The company says ASTRA supports some of the largest GPU fleets in the world, including its own inference fleet.
TSA is included free with every Together GPU Cluster. Users can enable it from the cluster Settings page, and Together AI says the service can be configured in under ten minutes. It works with existing S3-compatible stores, while T4 offers an option for keeping warm data next to the GPUs. T4 remains in Limited Preview.


