How to Choose a Vector Database by Interaction Pattern

This is a Journal Club piece. Its sibling is "RLM vs RAG." Part 1 covers why benchmarks mislead you, the seven dimensions that actually decide the pick, and the first five interaction patterns: one-shot Q&A, single-shot RAG, multi-turn chat, iterative/corrective retrieval, and agentic loops. Part 2 covers the rest, plus the full decision table, chunking, and the FAQ.

The benchmark trap

Most vector database picks start with a benchmark page. They end with the benchmark's winner. That is the wrong place to start.

A benchmark tells you queries per second at a given recall, on one fixed dataset. It does not tell you how many times your system retrieves per user turn: once, or seventeen. It does not tell you if you write ten thousand vectors a day or ten million. It does not tell you if your queries are short user questions or long prompts an agent assembled. The interaction pattern decides more than the benchmark does.

A Qdrant benchmark page and a Tiger Data (Timescale) pgvector comparison can both be accurate and still point you the wrong way. They answer "which engine is fastest on this synthetic workload." You need the answer to a different question: "which engine survives the way my product actually queries it." The first question has a clean number. The second has a shape. This piece reads the shape.

What drives the choice: the interaction pattern

The interaction pattern is the sequence and texture of how a user, an agent, or a pipeline touches retrieval. Different patterns stress different parts of a database. Seven dimensions matter more than raw throughput.

Recall sensitivity. How badly does a missed chunk hurt? A legal research tool pays more per query to push recall up. A product search box trades recall for speed and survives fine.

Latency budget. How much time can retrieval spend before the user notices? A chat interface tolerates 200 to 400 milliseconds. Autocomplete tolerates 50. An overnight batch job tolerates minutes.

Freshness. How fast does a new document need to be findable? A support knowledge base can ingest overnight. A trading-floor news system needs writes visible in seconds.

Query complexity. Is the query a single embedding lookup? A hybrid lexical-plus-dense fusion? A filtered search with structured predicates? A multi-hop traversal? This is where engines diverge most.

Write volume. How many vectors land per second at peak? Agentic and code-generation systems can write more vectors than they read. That inverts the economics of most cloud pricing.

Re-retrieval frequency. How many times does one user goal query the index? One-shot RAG queries once. A corrective loop queries two to five times. An agent with tools can query dozens.

Cost per query. What does one completed user interaction cost in retrieval terms? Pinecone serverless lists $16 per million reads, $4 per million writes, and $0.33 per gigabyte-month of storage. Multiply by the re-retrieval frequency before you compare that to self-hosted Qdrant or pgvector.

These seven dimensions, mapped against nine interaction patterns, produce a decision shape. The rest of this piece walks the patterns and lands the table in Part 2.

Stateless retrieval: one-shot Q&A and single-shot RAG

For one-shot Q&A and single-shot RAG, almost any modern vector database works. The choice comes down to what you already run. This is the easy case, and it gets the most attention because every demo uses it.

One-shot Q&A: a user asks a question, the system embeds it, retrieves the top-k chunks, and a model answers. No second turn, no agent, no re-retrieval. Single-shot RAG is the same shape with a bigger context window and a more deliberate prompt assembly step. Both patterns are read-heavy. Both are latency-sensitive, in the hundreds of milliseconds. Both tolerate stale data measured in hours or days.

If you already run Postgres, pgvector is the obvious default. It handles tens of millions of vectors competitively, with caveats the engineers will not let me skip. The "tens of millions" claim holds only with HNSW indexing, enough RAM to keep the index hot, and an ef_search value tuned to your recall target. The break-even point between pgvector and a dedicated engine like Qdrant or Weaviate is not a vector count. It is workload shape: query concurrency, filter selectivity, write churn, and how often you rebuild the index. The Tiger Data (Timescale) pgvector versus Qdrant benchmark is useful, but it is vendor-adjacent. Worth saying.

If you do not already run Postgres, choose between a managed service like Pinecone and a self-hosted engine like Qdrant or Weaviate. Pinecone serverless is a credit-card decision: priced per read, per write, per gigabyte-month, no operations burden, and a freshness profile that suits stateless retrieval. Qdrant gives you more control over the recall-latency curve and a lower per-query cost at scale, at the cost of running it yourself. Weaviate sits between them, with strong hybrid-search ergonomics that matter more in the next pattern than this one.

The anti-pattern here is reaching for a graph database or a GraphRAG-style pipeline. Neither earns its complexity when the user asks one question and leaves.

Multi-turn chat: filtered retrieval, not prompt caching

For multi-turn chat, you want an engine with fast filtered queries. You also want to stop calling prompt caching "retrieval." They solve different problems.

Multi-turn chat looks like one-shot Q&A within any single turn. The difference shows up across turns. A user's third question depends on the first. The system replays the conversation into the prompt, retrieves fresh chunks per turn, or does both. Latency budgets tighten, because the user is waiting. Filters matter, because the system often scopes retrieval by user, conversation, or document set.

This is where engineers reach for prompt caching and call it retrieval. It is not. Anthropic's prompt caching, documented in its August 14 and December 17, 2024 updates, lets you reuse a long static prefix across requests so the model does not reprocess it. OpenAI shipped the same shape on October 1, 2024. The mechanism is context reuse, not search. Anthropic's default TTL is five minutes, with a one-hour extended option. OpenAI's is roughly five to ten minutes. Both require an exact prefix match. Caching makes a system with a long, stable system prompt cheaper. It does not help you find the right document.

The right tool for multi-turn chat is a vector database with strong metadata filtering and decent hybrid search. Qdrant, Weaviate, and Pinecone all qualify. Weaviate's hybrid search got more ergonomic in version 1.24, which added rankedFusion and relativeScoreFusion. MongoDB Atlas added $rankFusion in 8.0 and $scoreFusion in 8.2, which makes it a credible choice if your application data already lives in MongoDB.

The word "hybrid" needs untangling before it causes more confusion. Hybrid retrieval, in production, almost always means BM25 plus dense vectors, fused with Reciprocal Rank Fusion. That is a query-time decision. Hybrid storage, where one engine holds both lexical and vector indexes, is a separate property. ParadeDB's pg_search puts Tantivy-based BM25 inside Postgres. Vespa has run hybrid retrieval and storage together for years. Vendor marketing blurs the two. It should not.

The anti-pattern is leaning on prompt caching as a substitute for real retrieval, then watching the first question the cached prefix did not anticipate land in a system with no idea what to do.

Iterative and corrective retrieval: what teams actually ship

For iterative and corrective retrieval, what teams run in production is a generic, LLM-judged re-retrieval loop, not the published Self-RAG or CRAG papers. The database has to handle two to five queries per user goal without blowing the latency budget.

The published methods are real. Asai and colleagues' Self-RAG, posted to arXiv in October 2023, trains a model with reflection tokens that decide when to retrieve, what to retrieve, and whether the retrieved chunks are relevant. Yan and colleagues' Corrective Retrieval Augmented Generation, posted January 2024, adds a lightweight evaluator that triggers a web search fallback when local retrieval falls short. Both are solid engineering. Both have public implementations.

What teams ship is rarely either paper. The common pattern: retrieve, ask the model if the context is sufficient, and if not, rewrite the query and retrieve again. Sometimes the rewrite decomposes into sub-questions. Sometimes it expands synonyms. Sometimes it falls back to "search the web instead," in the CRAG spirit. Calling this Self-RAG or CRAG overclaims. It is vibes-RAG with a retry loop. That's fine. The citation should just be honest.

The database implication is real. If a user goal triggers two to five retrieval calls, your effective queries-per-interaction is two to five times the headline number. Pinecone's per-read price gets multiplied by that same factor before you have paid for anything else. Self-hosted Qdrant or Weaviate amortizes better at this volume. pgvector survives if the index fits in RAM and query concurrency stays modest. Turbopuffer, which stores its vector and BM25 indexes natively on object storage like S3, gets interesting here, because its cost model assumes a low queries-per-document ratio and cheap storage. For a corpus that gets retrieved against often, that math reverses.

The anti-pattern is running a corrective loop against an engine priced for one-shot reads, on a corpus large enough that storage costs dominate, and finding out at the end of the month that retrieval costs more than inference.

Agentic loops and code generation: write volume and filters win

For agentic systems and code-generation tools, write volume and metadata filtering matter more than raw vector search speed. Retrieval here is a tool the agent calls, not a context-injection step before the prompt.

Anthropic's December 19, 2024 piece, "Building Effective Agents," frames this cleanly. An agent loop is a model that picks a tool, calls it, reads the result, and decides again. Retrieval is one tool among several. The agent might call a vector search, then a SQL query, then code execution, then another vector search scoped by a different filter. The pattern is bursty, scoped, and frequent.

Code generation against a codebase is the same shape with a sharper edge. Cursor, Sourcegraph Cody, and similar tools index source files into a vector database, then issue scoped retrievals as the developer types. Two things happen here that one-shot RAG never deals with. First, the codebase changes constantly. Every commit invalidates some embeddings and adds new ones. Write volume rivals or beats read volume. Second, queries are heavily filtered by language, file path, repository, and recency. The vector search is the easy part. Filter performance and write throughput are the hard parts.

This is where engines with strong filtering and fast incremental indexing pull ahead. Qdrant's filtered HNSW is built for it. LanceDB, built on the columnar Lance format and backed by S3, has been reported by AWS's Architecture Blog to handle billion-scale vector workloads, with column-oriented filters that map well to code metadata. Turbopuffer's object-storage-native design fits the "many corpora, low queries-per-document" shape of multi-repo code search.

The anti-pattern is running an agentic workload on a vector database without first-class metadata filtering. You will spend more on read-side filtering than on the vector search itself, and the queries-per-goal multiplier will make that impossible to ignore.

Two things worth keeping separate when you reason about agents and retrieval. One: retrieval-as-tool is not context-injection. The agent decides to query. The system does not pre-query. Two: write amplification is real and underdiscussed. An agentic system that summarizes its own past actions into memory writes back into the same store it reads from. Pricing built for "mostly read" workloads breaks down.

Part 2 covers the remaining patterns (structured extraction, long context, global sensemaking), the full decision table, chunking and embeddings, the open questions, and the FAQ. Continue to Part 2 →