How to Choose a Vector Database by Interaction Pattern, Part 2: The Decision Table

This is Part 2. Start with Part 1 for the benchmark trap, the seven dimensions, and the first five interaction patterns. Part 2 covers structured extraction, long context, global sensemaking, the full decision table, chunking and embeddings, open questions, and the FAQ.

Structured extraction and long context: when to skip retrieval

For structured extraction and long-context workflows, the failure mode is degradation inside the context window, not a database choice. The Chroma Context Rot report is the best primary source on what actually breaks.

Structured extraction is the pattern where a model reads a document and produces typed output: invoice fields, contract clauses, regulatory disclosures. The real question is whether you need a vector database at all. For a single document under the model's effective context, the answer is usually no. Load the document, prompt for the schema, parse the output. Retrieval shows up in deduplication, similarity to past extractions, or routing to schema variants. These are small, well-bounded indexes. Any vector database works, and your engineering attention belongs elsewhere.

The long-context "just stuff it all in" pattern deserves more skepticism than it gets. Frontier model context windows now reach into the hundreds of thousands of tokens. The temptation is to skip retrieval and load the whole corpus. Two pieces of evidence make that expensive in practice.

The Chroma Context Rot report, published July 2025, tested 18 models across input lengths and found non-uniform degradation. Performance dropped well before the advertised context limit, and the drop was not uniform across tasks. Many models showed an effective working context closer to 300 to 400 thousand tokens than to their nominal one to two million. The needle-in-a-haystack test, used for years to claim long-context wins, holds up better than harder tasks because it is a single-fact lookup. Multi-hop reasoning, where the answer needs facts combined from across the document, breaks much earlier, often in the 32 to 128 thousand token range on RULER-class and LongBench-class evaluations. The Chroma report states this most clearly.

The practical consequence: for any task that reasons across the loaded content, retrieval still earns its keep below the advertised context limit. The database choice here matches one-shot RAG. The real question is whether to retrieve at all, and the answer is more often than you think.

The anti-pattern is trusting the model's nominal context window as its useful context window, then watching the multi-hop questions fail at evaluation time.

Global sensemaking: when a vector database is the wrong tool

For global sensemaking, where the user wants a synthesis across a whole corpus rather than an answer from a few chunks, a vector database is the wrong primary tool. GraphRAG is the right shape, with real limits.

Microsoft Research's "From Local to Global: A Graph RAG Approach to Query-Focused Summarization," posted to arXiv in April 2024 and shipped as the Microsoft GraphRAG repository, makes the case well. Three steps. First, extract entities and relationships from the corpus into a knowledge graph. Second, run Leiden community detection on the graph and have a model summarize each community. Third, at query time, map-reduce across the community summaries to assemble an answer.

This wins on questions like "what are the major themes in this corpus" or "summarize the relationships between these three actors." It loses on questions like "what does this specific document say about this specific clause." Vector search wins the local lookup. GraphRAG wins the global synthesis. They are not competing products. They solve different questions.

Neo4j's neo4j-graphrag-python package gives you a production-ready way to run this on Neo4j: the graph database handles storage and traversal, a model handles extraction and summarization. Two honest limits. First, extraction quality dominates the whole pipeline. On clean, structured corpora it works well. On noisy corpora, like raw web scrapes or transcribed audio, entity extraction errors compound through the community summaries. Second, the build cost is real. Generating community summaries with a large model across a million-document corpus is not free, and you pay it up front, before anyone queries anything.

The anti-pattern is using a vector database for a question that needs synthesis. The user asks "what are the recurring patterns across these ten thousand customer interviews," and the system retrieves the top twenty interviews and summarizes them. The answer reflects the top twenty, not the ten thousand. GraphRAG, or some other map-reduce-over-summaries shape, is the right tool.

The decision table

This matrix maps the nine patterns to primary, alternative, and anti-pattern picks. Real database names in each cell. Use it as a starting point, not a verdict. Two levers matter more than which row you land on: chunking strategy and the embedding model. Reranking sits across every row and is rarely optional in production. Both get their own section after the table.

A note on the cells. Pinecone serverless is excellent at low-to-medium query volumes, where operational simplicity is worth the per-read cost. It gets harder to justify in the corrective and agentic rows, because the read multiplier flips the unit economics. That's a pricing-shape observation, not a verdict on the product. Turbopuffer is interesting wherever storage cost dominates and queries-per-document is low, which fits multi-corpus and infrequently-accessed archives more than hot interactive workloads. LanceDB earns the code-generation slot because the columnar Lance format handles code metadata filters cleanly, and S3 backing keeps storage affordable as the index grows.

Embedding model and chunking: the two levers bigger than the database

The two levers that matter more than which vector database you pick are the embedding model and the chunking strategy. Reranking is not optional in any production stack that cares about quality.

Anthropic's "Introducing Contextual Retrieval" piece, published September 19, 2024, gives the cleanest public numbers. Headline: Contextual Retrieval cuts retrieval failure by 49 percent. Stacked with a reranker, it cuts failure by 67 percent. The numbers need precision. They measure retrieval failure at the top-20 chunks, not answer accuracy. The 35 percent reduction is Contextual Embeddings alone. The 49 percent is Contextual Embeddings plus Contextual BM25. The 67 percent needs Contextual Embeddings plus Contextual BM25 plus a reranker, all three stacked. The reranker carries a lot of that win.

Chunking strategy sits upstream of all of this. Anthropic's Contextual Retrieval works by prepending a model-generated, chunk-specific context to each chunk before embedding it. That single change does more work than swapping vector engines does. The principle: a chunk needs enough self-contained meaning to be retrievable on its own. Sliding-window chunking with no overlap and no context preamble produces embeddings that are technically correct and operationally useless, because no chunk knows what document or section it came from.

The embedding model is the other lever. A modern general-purpose embedding model on a clean, well-chunked corpus beats a weaker model on the same corpus, no matter which vector engine sits underneath. The engine is a substrate. The embedding model is the signal.

Reranking deserves its own paragraph, because vendor pitches call it optional and it isn't. A cross-encoder reranker takes the top-50 or top-100 results from your vector search and rescores them by reading the query and each candidate together. That costs more per candidate than the bi-encoder vector search, which is why it runs second. Production stacks that care about answer quality almost always run a reranker. Calling it optional is wrong. Cohere, Voyage, and open-weights rerankers are all credible options. The bigger question is whether you run one, not which one.

Open questions

A few honest gaps I could not close inside this piece.

The pgvector scaling threshold has no clean peer-reviewed source. The "tens of millions of vectors" number gets repeated across vendor and vendor-adjacent benchmarks, including Tiger Data's pgvector versus Qdrant comparison. No independent academic measurement of the break-even point exists as a function of workload shape. The right answer is probably "test on your workload," but that isn't a citation.

The claim that hybrid retrieval beats pure vector retrieval is widely repeated and consistent with Anthropic's Contextual Retrieval numbers. I have not found a single peer-reviewed meta-analysis that establishes it across retrieval tasks. The evidence is a stack of vendor docs and individual studies, each defensible, none synthetic across the field.

There is no apples-to-apples cross-vendor cost table for vector databases at production scale. Pinecone serverless publishes per-read, per-write, and per-storage rates. Self-hosted Qdrant or Weaviate costs depend on instance choices, redundancy, and operations time. Managed Weaviate Cloud and Qdrant Cloud publish their own rates. A credible head-to-head needs assumptions every vendor will dispute. This piece deliberately skips putting one number on it.

There is no measured backend survey for agentic stacks. The claim that Qdrant and LanceDB dominate agentic and code-generation deployments rests on documented architectures from Cursor, Sourcegraph, and other public stacks. No clean industry-wide measurement exists of which vector databases sit under production agents today.

If you have primary sources that close any of these gaps, send them. I'll update.

FAQ

Is pgvector good enough for production RAG? Yes, for many workloads, with caveats. pgvector with HNSW indexing, enough RAM to keep the index hot, and a tuned ef_search parameter handles tens of millions of vectors at production quality. The break-even point against a dedicated engine like Qdrant is workload-shaped, not vector-count-shaped. High query concurrency, low filter selectivity, or frequent index rebuilds push you toward a dedicated engine. If you already run Postgres and the workload is moderate, pgvector saves you an entire service.

What is the difference between hybrid retrieval and hybrid storage? Hybrid retrieval is a query-time technique: run a lexical search like BM25 and a dense vector search in parallel, then fuse the results, usually with Reciprocal Rank Fusion. Hybrid storage is a system property: one engine holds both lexical and vector indexes. Vespa, Weaviate (since v1.24), and MongoDB Atlas (with $rankFusion in 8.0 and $scoreFusion in 8.2) offer hybrid storage. You can run hybrid retrieval across two separate engines, like Elasticsearch and Pinecone, without hybrid storage. Vendor marketing conflates the two.

Does prompt caching replace retrieval? No. Prompt caching, documented by Anthropic in August and December 2024 and by OpenAI in October 2024, lets you reuse a long static prefix across requests so the model doesn't reprocess it. The TTL is short: around five minutes for Anthropic's default, up to one hour with the extended option, and roughly five to ten minutes for OpenAI. Both require an exact prefix match. Caching makes a system with a long, stable system prompt cheaper. It doesn't let you search a corpus or surface the right document.

When should I use GraphRAG instead of a vector database? When the question needs synthesis across the corpus, not a lookup from it. Microsoft's GraphRAG, published April 2024, uses entity extraction, Leiden community detection, model-generated community summaries, and map-reduce at query time. It wins on questions like "what are the major themes" or "summarize the relationships between these actors." It loses on questions like "what does this clause say." Vector search wins the local lookup. The two are complementary, for different question shapes.

What do the Anthropic Contextual Retrieval numbers actually mean? They measure retrieval failure at the top-20 chunks, not answer accuracy. The 35 percent reduction is Contextual Embeddings alone. The 49 percent is Contextual Embeddings plus Contextual BM25. The 67 percent needs all three: Contextual Embeddings, Contextual BM25, and a cross-encoder reranker stacked together. The reranker carries a large part of the gain. Read the piece in full. The numbers are precise and easy to misquote.

Is the long-context window large enough that I can skip retrieval? For single-fact lookup, often yes. For multi-hop reasoning, usually no. The Chroma Context Rot report, published July 2025, tested 18 models and found non-uniform degradation across input lengths and tasks. Effective working context for many models lands around 300 to 400 thousand tokens, well below their nominal one to two million. The needle-in-a-haystack test holds up better than harder evaluations, because it tests a single-fact lookup. Multi-hop reasoning breaks much earlier, often in the 32 to 128 thousand token range on RULER-class and LongBench-class evals.

Are teams actually running Self-RAG and CRAG in production? Rarely the literal papers. Self-RAG (Asai et al., October 2023) and CRAG (Yan et al., January 2024) are real published methods with public implementations. What teams ship is usually a generic loop: retrieve, ask the model if the context is sufficient, rewrite and retry if not, with a web search or alternate-source fallback in the CRAG spirit. Calling that Self-RAG or CRAG overclaims. The retrieval implication holds either way: two to five retrievals per user goal changes the unit economics.

How do I think about Pinecone's pricing versus self-hosting? Multiply the headline read and write rates by your re-retrieval frequency before you compare. Pinecone serverless lists $16 per million reads, $4 per million writes, and $0.33 per gigabyte-month of storage. A one-shot RAG application at one read per user interaction is cheap. An agentic application at twenty retrievals per goal multiplies that read cost by twenty. Self-hosted Qdrant or Weaviate amortizes better at high query volumes. Turbopuffer's object-storage-native model fits low-queries-per-document corpora. Compare per completed user interaction, not per query.

Sources

Anthropic, "Introducing Contextual Retrieval," September 19 2024. https://www.anthropic.com/news/contextual-retrieval

Anthropic, "Building Effective Agents," December 19 2024. https://www.anthropic.com/research/building-effective-agents

Anthropic, "Prompt caching with Claude," August 14 2024 and December 17 2024 updates. https://www.anthropic.com/news/prompt-caching

OpenAI, "Prompt Caching in the API," October 1 2024. https://openai.com/index/api-prompt-caching/

Edge, Trinh, Cheng, Bradley, Chao, Mody, Truitt, Larson, "From Local to Global: A Graph RAG Approach to Query-Focused Summarization," arXiv:2404.16130, April 2024. Microsoft GraphRAG repository: https://github.com/microsoft/graphrag

Chroma, "Context Rot: How Increasing Input Tokens Impacts LLM Performance," Technical Report, July 2025. https://research.trychroma.com/context-rot

Asai, Wu, Wang, Sil, Hajishirzi, "Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection," arXiv:2310.11511, October 2023.

Yan, Gao, Li, Sun, Wang, Chen, Wang, "Corrective Retrieval Augmented Generation," arXiv:2401.15884, January 2024.

Tiger Data (Timescale), "Pgvector vs. Qdrant" benchmark. https://www.tigerdata.com/blog/pgvector-vs-qdrant

Pinecone serverless pricing. https://www.pinecone.io/pricing/

Qdrant benchmarks page (https://qdrant.tech/benchmarks/); Weaviate hybrid search docs covering rankedFusion and relativeScoreFusion since v1.24 (https://weaviate.io/developers/weaviate/search/hybrid); Vespa hybrid retrieval tutorial (https://docs.vespa.ai/); MongoDB Atlas $rankFusion in 8.0 and $scoreFusion in 8.2 (https://www.mongodb.com/docs/atlas/atlas-vector-search/)

Turbopuffer, object-storage-native vector and BM25 search. https://turbopuffer.com/. LanceDB, columnar Lance format, S3-backed billion-scale per AWS Architecture Blog. https://lancedb.com/ and https://aws.amazon.com/blogs/architecture/

Neo4j neo4j-graphrag-python package. https://neo4j.com/docs/neo4j-graphrag-python/. ParadeDB pg_search, Tantivy-based BM25 in Postgres. https://www.paradedb.com/

Luke Paxton. Square Mile Design, May 2026.