Artificial intelligence news.
Straight from the original announcements.

LanceDB Adds GPU Acceleration for Large-Scale Vector Indexing

November 19, 2026, 08:16By the AI News Bites Editorial TeamSource: LanceDB
LanceDB Adds GPU Acceleration for Large-Scale Vector Indexing
Image: LanceDB

LanceDB has added GPU acceleration for vector index training, allowing users with Nvidia GPUs or supported Apple Silicon devices to use CUDA or MPS when building indexes. The company says the change can make KMeans training up to 20, 26x faster than CPU-based processing.

Key points
  • LanceDB v0.3.3 adds CUDA and MPS support for index training.
  • GPU acceleration targets KMeans training for IVF indexes.
  • LanceDB reports up to 20, 26x faster training than CPU processing.
  • The benchmark used 1-million 1536D vectors and IVF_4096.
  • PQ training and vector assignment acceleration are planned next.

GPU support targets a costly indexing bottleneck

Vector databases support applications including retrieval-augmented generation, recommendation systems and computer vision. While much of the discussion around them focuses on query latency and recall, LanceDB says the time required to build a vector index can become a major bottleneck as datasets grow.

Building an index is computationally intensive, and the company says the work increases quadratically with the number of vectors or their dimensions. Common techniques such as IVF, or Inverted File index, and PQ, or Product Quantization, divide vectors into clusters. Finding the cluster centroids requires KMeans, an iterative process involving large numbers of vector distance calculations.

LanceDB says GPUs are well suited to this type of computation. Its latest Python package release, LanceDB v0.3.3, backed by Lance v0.8.10, adds GPU acceleration through an accelerator parameter in create_index().

CUDA and MPS are available through the Python package

The implementation supports Nvidia GPUs through CUDA and Apple Silicon GPUs through MPS. Under the hood, LanceDB uses PyTorch to train IVF clusters, then passes the resulting KMeans centroids to its Rust core for index serialization.

The company says this approach provides support for two major developer platforms, Linux and Mac. Users need to install PyTorch, including a CUDA-enabled version where applicable, to use the GPU features.

LanceDB also points to other indexing improvements, including out-of-memory shuffling and batched KMeans training on GPUs. According to the company, these changes allow the system to train indexes on tens of millions of vectors while reducing concerns about CPU or GPU out-of-memory failures.

Tests show faster KMeans training

For its benchmark, LanceDB trained an IVF_4096 index containing 4096 clusters with L2 Euclidean distance. The test used 1-million vectors based on OpenAI Ada2 embeddings with 1536 dimensions, running on both Linux and macOS.

On Linux, the test used a Google Cloud g2-standard-32 instance with 32 logical cores and one Nvidia L4 GPU, running Ubuntu 22.04 with nvidia-driver-525. CPU training took 323 seconds, compared with 12.5 seconds on the GPU.

The macOS test used a 14-inch Macbook Pro with an Apple M2 Max, 64GB of RAM and 30 GPU cores. Training took 397 seconds on the CPU and 21 seconds on the GPU. LanceDB reports that GPU acceleration delivered up to 20, 26x speedups compared with its CPU counterparts.

More index stages are planned for acceleration

The current release accelerates IVF KMeans training, which is only one part of the full index training process. LanceDB says it plans to add GPU acceleration for PQ training and for assigning vectors to their correct centroids. The company expects those additions to produce further improvements in end-to-end index training time.

Because the implementation uses PyTorch, LanceDB says it also has a path toward distributed GPU training and future support for additional hardware accelerators, including TPUs through XLA. CUDA and Apple Silicon GPU support is available through pip install lancedb, giving developers a way to test the feature on compatible hardware now.

Related news

Lance introduces community governance for its open format
For developersNovember 19, 2026, 08:16

Lance introduces community governance for its open format

Lance has introduced a formal governance structure for its open format community, defining contribution paths, leadership roles and consensus-based decisions. The…

Lance Data Viewer Offers a Lightweight UI for Exploring Lance Tables
For developersNovember 19, 2026, 08:16

Lance Data Viewer Offers a Lightweight UI for Exploring Lance Tables

Lance Data Viewer is an open source web interface for browsing Lance datasets, inspecting schemas and reviewing table contents. Built by Gordon Murray as a debugging…

LanceDB proposes multi-base paths for distributed Lance datasets
For developersNovember 16, 2026, 18:17

LanceDB proposes multi-base paths for distributed Lance datasets

LanceDB has introduced Lance’s multi-base path model, a design aimed at distributing one dataset across multiple storage locations without giving up the portability of…