Artificial intelligence news.
Straight from the original announcements.

LanceDB Combines Lance Namespace and Ray for Scalable AI Pipelines

November 19, 2026, 08:16By the AI News Bites Editorial TeamSource: LanceDB
LanceDB Combines Lance Namespace and Ray for Scalable AI Pipelines
Image: LanceDB

LanceDB has presented an end-to-end approach for production AI workloads that combines Lance Namespace, Ray and LanceDB. The company says the integration connects existing metadata services with distributed data processing, vector search and full-text search.

Key points
  • lance-ray is now an independent subproject for distributed Lance operations.
  • Lance Namespace provides native Python and Rust SDKs.
  • The example processes the BeIR/quora dataset with Ray and LanceDB.
  • LanceDB supports vector, full-text and hybrid search in the workflow.

Three components for production AI workloads

The approach uses Lance Namespace to integrate Lance tables with enterprise metadata services, Ray for distributed ingestion and feature engineering, and LanceDB for vector and full-text search. According to LanceDB, the combination is designed to help build AI applications that work with existing infrastructure while supporting the scale required for real-world deployments.

The lance-ray package has become an independent subproject. It supports distributed read, write and data evolution operations on Lance datasets through Ray’s parallel processing capabilities. This is intended for large-scale data transformations and feature engineering across a compute cluster.

Lance Namespace also now includes native Python and Rust SDKs. LanceDB says these SDKs enable integration across programming languages and support connections with both lance-ray and LanceDB.

An end-to-end question-answering example

LanceDB demonstrates the workflow with the BeIR/quora dataset from Hugging Face. The example loads the corpus, converts it into a Ray Dataset and writes it to Lance format through Lance Namespace. The sample uses the directory-based namespace, while also showing configurations for Hive Metastore, AWS Glue Catalog and Unity Catalog.

The example then uses the BAAI/bge-small-en-v1.5 Sentence Transformers model to generate embeddings. Ray adds the resulting vector column through distributed processing, using batches of 100 and a concurrency setting of 4. The example also allows GPU resources to be passed to Ray workers when GPUs are available.

After feature engineering, the same dataset is opened through LanceDB. The example creates a cosine vector index using the IVF_PQ index type and runs a semantic search for the question, “How do I learn machine learning?” It also creates a full-text index and searches for “machine learning algorithms.”

Use cases and next steps

LanceDB identifies several possible applications for this integration pattern, including retrieval-augmented generation applications, recommendation systems, multimodal search, feature stores and real-time analytics. The company also says LanceDB supports hybrid search, combining the semantic capabilities of vector search with keyword matching.

For more advanced feature engineering, LanceDB points to Geneva, its feature engineering framework. The company describes Geneva as providing schema enforcement, versioning and complex transformations, alongside capabilities such as lazy materialization, partial backfill and fault-tolerant execution.

The getting-started instructions call for installing lance-ray and lancedb, then consulting the Lance-Ray, LanceDB and vector search documentation. For users, the announcement provides a route from distributed data ingestion and embedding generation to searchable AI data, while enterprises can choose among multiple metadata services through Lance Namespace.

Related news

Lance introduces community governance for its open format
For developersNovember 19, 2026, 08:16

Lance introduces community governance for its open format

Lance has introduced a formal governance structure for its open format community, defining contribution paths, leadership roles and consensus-based decisions. The…

Lance and DuckDB Bring Vector and Hybrid Search Into SQL
For developersNovember 19, 2026, 08:16

Lance and DuckDB Bring Vector and Hybrid Search Into SQL

Lance has introduced a DuckDB extension that brings vector, full-text and hybrid retrieval directly into SQL queries. The integration is designed to keep multimodal…

Lance File 2.1 targets smaller storage and faster random access
For developersNovember 19, 2026, 08:16

Lance File 2.1 targets smaller storage and faster random access

Lance File 2.1 has entered beta with a redesigned encoding system intended to reduce file size and improve access to individual values. Lance says the update builds on…