We need RAG - buy a vector DB?

Same shelf, different job…

A2 · 420 × 594 mm · print-first

This poster separates two things that get sold as one: a vector database (infrastructure for similarity search) and RAG (a retrieve-then-generate pattern). Buying an index is not the same as grounding an answer.

The headline is the common mistake — “we need RAG, so buy a vector DB.” Sometimes you need the store without the generation loop; sometimes you need RAG with keyword or hybrid retrieval and no dedicated vector product. Related method terms: ANN indexes, Adaptive / funnel retrieval, Matryoshka embeddings.

The confusion

People name the pattern — and buy the store

Teams say they need RAG and immediately shortlist Pinecone, Weaviate, pgvector, Qdrant, and peers. That shopping list is about storage and approximate nearest-neighbour search under Vector DBs.

A vector database is infrastructure. RAG is a loop: retrieve relevant material, put it in the prompt, generate. The loop may use a vector DB — or BM25, hybrid search, graphs, or plain SQL. Do not confuse either with long-lived Memory.

Vector DB = store · index · search
RAG = retrieve · ground · generate
They are not the same object

Vector database

Infrastructure — embeddings in, neighbours out

Documents (or rows) are chunked, embedded into vectors, and written into an index (HNSW, IVF, disk-backed variants, and filters — see ANN indexes). Queries return top-k ids, scores, and payloads. Packs: FAISS, Chroma, Milvus, Qdrant, Weaviate, pgvector.

The database owns storage, ANN search, metadata filters, and scale. It does not write the natural-language answer. An embedding model is required to get into vector space; a generative LLM is not required for search itself. Upcoming sheet: Embedding and Chunking.

Documents → Embed → Vector index → Top-k hits
Output: ids · scores · payloads — not prose

RAG

Pattern — retrieve → ground the prompt → generate

Retrieval-Augmented Generation fetches relevant passages for a question, augments the prompt with them, then lets the model generate — ideally with citation paths back to the retrieved material. Frameworks such as LangChain, LlamaIndex, and Haystack often own this loop.

RAG owns grounding and the “less invented facts” path. It needs a retriever behind it; that retriever is often a vector database, but it does not have to be. Compare weight-side learning under Fine-tuning vs RAG / Fine-tuning.

Question → Retrieve → Augment → Generate → cite
Retriever may be vector, keyword, hybrid, or mixed

One store, two flows

Shared shelf — ranked hits vs prose + cite

Both sides read the same store (Lisbon · Reykjavik · Tokyo). The left flow embeds the question, hits the vector DB, and stops at ranked cities and scores. The right flow retrieves, builds a prompt with the hit ids, then generates a short answer with a cite.

Compact lens: vector DB out = cities · scores · stop (no LLM). RAG out = prose answer · cite (needs LLM). Funnel / multi-pass retrieval sits under Adaptive / funnel retrieval.

Do not confuse your investor!

Same shelf, different job

Closing line only. The travel example already shows what each side returns: an id table of cities + scores, versus one written answer with cite id 01 Lisbon.

Same shelf in the diagram, different job in the system. Do not buy a vector database and call it RAG — or skip the store and expect grounded cites. See also Quantization for bit-width tradeoffs and Evals that matter for measuring whether grounding actually worked.

Sources

Lewis et al., Retrieval-Augmented Generation (RAG)
ANN indexes: HNSW / IVF family
happyoverlord.com research posters

Propose a topic

Suggest the next AI infographic. One topic, one glance — what should it make obvious?