Open-Source Vector Databases Compared for RAG Workloads
Match each RAG workload to the right open-source vector database.

The choice of an open-source vector database for retrieval-augmented generation (RAG) depends on the workload, so no one tool wins for every workload. Scale, filtering needs, existing infrastructure, and hybrid-search requirements point to different tools. This comparison explains how those variables map to six options, and why the best choice is the one that solves the retrieval problem without creating an unnecessary operations problem.
Why no single open-source vector database wins every RAG workload
An engineer choosing a production database has to answer two questions at once. Can it retrieve the right documents, and can the team run it within its operational limits? Selective filters can leave too few results that the application can use, and a distributed engine can add more overhead than a small application needs.
The options fall into two architectural families. Qdrant, Weaviate, Milvus, and Chroma are purpose-built vector engines, while pgvector adds vectors to Postgres and Redis combines vector search with caching and operational data. Index type, filtering stage, and distribution model explain which family, and which tool within it, fits a particular workload.
The four variables that actually determine which database fits a workload
Scale starts with the number of vectors. At smaller counts, avoiding another database often outweighs differences in index performance. As counts grow, HNSW parameters, quantization, and distributed sharding become more important. At hundreds of millions to billions of vectors, distribution and GPU acceleration can become hard requirements.
Filtering complexity asks how much of that data each query can use. An approximate nearest-neighbor search followed by metadata filtering can discard most of its results when predicates are selective. Recall then collapses. The caller gets fewer results that it can use than it asked for and must over-fetch and discard, while one-stage filtering checks the predicates during index traversal.
Product search, legal retrieval, and enterprise knowledge bases may need to find exact terms as well as semantic matches. Dense vectors do not replace every keyword query. A requirement for BM25 keyword matching, dense similarity, and metadata filtering in one query therefore narrows the field differently from a requirement for vectors alone.
Existing infrastructure asks what the team would have to add. A Postgres team takes on new operational work by introducing a dedicated vector store, while a Kubernetes team without a relational database takes on work by adding Postgres solely for pgvector. Redis offers another path. When caching and operational state are also required, consolidation may be more useful than a separate vector service.
Qdrant: the default for filter-heavy, self-hosted production RAG
Selective metadata filtering is the clearest reason to start with Qdrant for a new self-hosted production pipeline. Its payload filtering is integrated into HNSW graph traversal. That changes which candidates the search considers. Instead of finding neighbors and then discarding those that fail the predicates, the search incorporates those restrictions, avoiding the post-filter recall collapse that undermines filtered retrieval.
Qdrant combines that mechanism with low latency and relatively low operational overhead. The Rust-based engine posts low single-digit millisecond p50 query latency in independent benchmarks, though that is not a promise for every workload. Quantization helps keep memory use manageable. The open-source version is free, and Qdrant Cloud has a free 1 GB cluster and bills its Standard-tier by the hour for the compute and memory used.
Its default status still has a boundary. At true billion-scale, distributed sharding across many nodes takes more work to set up by hand than it does with Milvus, which has a more established distributed architecture. A team reaching that scale should evaluate Milvus rather than assume the same operational balance will hold.
Weaviate: when hybrid search must happen in a single native query
What changes when an exact keyword match is essential? The priority shifts from filtering alone to combining retrieval methods without external orchestration. Weaviate natively combines dense vector similarity, BM25 keyword matching, and metadata filtering in one query. That makes it a strong open-source choice when hybrid retrieval is a hard requirement.
Against Qdrant’s filtering-led baseline, Weaviate’s distinguishing feature is the query architecture. Native hybrid search needs less external glue, and built-in vectorization can cut the work needed to integrate embeddings. Its multi-tenancy also provides mature isolation for SaaS-style deployments with separate customer knowledge bases. These are different advantages from low vector-search latency.
Weaviate supports HNSW, Flat, Dynamic, and HFresh indexes under the BSD-3 license. It is not the outright fastest option at extreme scale, and billion-vector workloads with demanding write throughput still warrant a Milvus evaluation. Administration has limits too. Some operations are restricted in the cloud management console, which is relevant for teams that depend on UI-based management.
Milvus: when scale reaches hundreds of millions of vectors or beyond
At hundreds of millions to billions of vectors, distribution becomes a selection trigger. Milvus is built for horizontally scaled retrieval, with a cloud-native distributed architecture commonly deployed on Kubernetes. This is its reason to enter the shortlist. It serves workloads whose volume exceeds the comfortable operating range of simpler deployments, especially when continuous high-volume ingestion must coexist with retrieval.
Milvus supports HNSW, IVF_FLAT, IVF_PQ, SCANN, and DiskANN indexes, as well as GPU-accelerated indexing. Those choices give large workloads more ways to balance indexing and retrieval demands. It also has particularly strong write throughput among the purpose-built engines in this comparison. Milvus is licensed under Apache 2.0 and is available through the managed Zilliz Cloud service.
The cost includes both infrastructure capacity and operational complexity. To run a self-managed distributed deployment, a team needs infrastructure engineers who can run Kubernetes and handle the work that comes with it. For smaller RAG applications, that work can cost more than the gain in performance is worth.
pgvector: the pragmatic starting point when Postgres is already running
An existing Postgres deployment changes the calculation before index performance enters it. pgvector is a PostgreSQL extension, so vectors can live next to relational data with ACID guarantees and standard SQL filtering. There is no separate database to provision, monitor, or secure. For many RAG workloads below dedicated-engine scale, keeping things this simple has little effect on how well retrieval works.
pgvector supports HNSW and IVFFlat indexes and multiple distance metrics under the PostgreSQL License. Its important constraint is the filtering stage. Metadata filters apply after the index scan rather than during graph traversal, so highly selective predicates can leave too few useful results. SQL-native filtering and filter-aware vector traversal are not the same capability.
Recall collapse caused by filtering, latency that is too high, or vector counts nearing the tens of millions are reasons to test a dedicated engine. They are not reasons to add one before those conditions arise. Starting with pgvector is an appropriately scoped infrastructure choice for a production architecture.
Chroma: prototype speed at the cost of production headroom
Chroma serves the earliest stage of that progression. Its simple Python API and local-first architecture offer a fast path to a working RAG prototype without provisioning a server. Demos, small projects, and internal tools fit those priorities. High-traffic and large-scale production workloads do not share the same requirements.
Chroma is licensed under Apache 2.0, but its hosted path is changing. Chroma announced on October 23, 2026 that it had signed a definitive agreement to join Replit and that Chroma Cloud would wind down. Writes end November 22, 2026, and reads end December 22, 2026. Before production traffic or multi-tenant requirements arrive, the next engine should be evaluated against the filtering, hybrid-search, and scale needs already described.
Redis: when vectors, caching, and operational data belong in one system
Some pipelines need more than a place to search embeddings. Session state, rate limiting, semantic caching, and real-time feature data introduce operational requirements that a standalone vector engine does not resolve. Redis addresses that system-count problem directly. Its memory-first platform can hold vectors, session data, counters, and application state in one system rather than requiring separate services for each role.
Redis Search provides vector storage and retrieval. Redis Iris builds on the same platform as a real-time context engine for AI agents, supporting memory, semantic caching, and governed data access. The index options include HNSW, FLAT, and SVS-VAMANA. Through FT.HYBRID, Redis combines vector similarity with filtering on geographic, numeric, tag, or text fields, while Redis 8.4 adds Reciprocal Rank Fusion and linear combination for score fusion.
The deployment choices reflect different operational preferences. Redis Cloud is fully managed without requiring Kubernetes, Redis Software is self-managed enterprise software, and Redis Open Source is free without requiring orchestration. Licensing varies by deployment across RSALv2, SSPLv1, and AGPLv3. Consolidation still needs a purpose. A team that needs only vector search and has no need for caching or operational data may find a leaner dedicated engine easier to run.
Beyond the index: what the vector database alone cannot determine about RAG quality
Chunking strategy and embedding quality shape what the index can find before a query reaches it. Reranking then affects which retrieved chunks deserve priority. Output evaluation checks the generated answer as well as the retrieved candidates, so things can go wrong in different ways at each stage.
If the bottleneck lies in chunking or the embedding model, a database migration leaves it intact. Speed and accuracy are different outcomes. Multi-stage pipelines that add cross-encoder reranking and a judge or evaluation layer address quality limits beyond single retrieval, for which roughly 72% answer accuracy is a cited limit.
Reviewable answers also require provenance through the pipeline. Associating each generated response with the specific document chunks that produced it is an architectural requirement that supports human review, source auditing, and compliance verification.
Applying the framework: a decision path from workload variables to tool
To turn these distinctions into a choice, start with infrastructure. If Postgres is already running, the dataset is under a few million vectors, and filters are not highly selective, pgvector is the practical starting point. pgvectorscale is another option beyond standard HNSW limits. If the application instead needs vectors together with caching, session state, or real-time operational data, evaluate Redis for the reduction in systems to manage.
Hundreds of millions to billions of vectors, continuous high-volume ingestion, and engineers available to operate Kubernetes point toward Milvus. Chroma fits a prototype or small internal tool, with a plan to migrate to production before traffic arrives.
For a new self-hosted production pipeline between those extremes, filtering becomes the next discriminator. Heavy metadata filtering makes Qdrant the default starting point, with Milvus entering the evaluation as true billion-scale distribution approaches. If native BM25 plus dense retrieval is a hard requirement, Weaviate becomes the stronger fit. Its built-in vectorization may also justify the choice when it meaningfully reduces integration work.
A team that outgrows pgvector’s filtering behavior can focus its reassessment on that variable. Migration is an engineering decision. The final choice still needs validation against the workload’s measured recall, acceptable latency, vector volume, and operational demands, because no general ranking can establish those limits for a particular application.


