RAG Infrastructure Requirements: A UK Buyer's BOM for 2026

The UK RAG Procurement Mistake: Funding the GPU, Not the Stack

Ask most UK IT leaders what a RAG deployment needs and the conversation starts and ends with GPUs — how many, what VRAM, which cloud instance. That instinct is understandable, because generation genuinely is GPU-heavy. But it skips the tier that vendor sizing documentation says actually determines whether your retrieval works at all: the RAM, disk and index headroom that sit underneath the vector store.

Qdrant's own capacity-planning guidance treats retrieval sizing as a distinct exercise: estimate RAM and disk per collection, then sum collections for the cluster total, starting from the number of points multiplied by the replication factor. Milvus publishes a dedicated sizing tool for the same reason — resource requirements and deployment configuration are calculated inputs, not GPU-adjacent defaults. Neither vendor frames vector search as something a generation-tier GPU budget happens to cover.

The practical implication for a UK buyer is simple but easy to miss in a vendor pitch: get a bill of materials that separates the retrieval plane (CPU, RAM, SSD, index overhead) from the generation plane (GPU, VRAM), and interrogate both. A quote that only itemises the accelerator has, by definition, not sized your RAG system.

Two Pipelines, Two Bottlenecks: Where CPU/RAM Ends and GPU Begins

A RAG pipeline is really two systems bolted together, and they fail in different ways. The retrieval tier — ingestion, chunking, embedding storage, index search, payload filtering, reranking — is bound by RAM capacity, disk I/O and index-to-memory ratios. The generation tier — the LLM that turns retrieved context into an answer — is bound by GPU VRAM and compute throughput.

This distinction matters for budgeting because the two tiers scale on different variables. Retrieval cost tracks vector count, embedding dimension and replication factor. Generation cost tracks concurrent requests, context length and model size. Sizing them as one line item, as many procurement conversations still do, means you either over-provision GPUs while starving the retrieval layer of RAM, or the reverse.

None of the public sizing documentation reviewed for this piece — from Qdrant, Milvus or the pgvector ecosystem — makes any GPU recommendation for the vector-store tier itself. That absence is itself the evidence: retrieval sizing is a CPU/RAM/disk problem, full stop, and generation sizing belongs in a separate conversation, ideally informed by tools built specifically for that job such as our AI GPU calculator.

The Vector-Index Memory Maths, Straight From the Vendors

Qdrant's formula for dense vector storage is dense_size = base × dimensions × bytes_per_dim — meaning the dimensionality of your embedding model is a first-order variable in your RAM bill, not a footnote you settle after choosing a model for accuracy reasons. On top of the core vector storage, Qdrant recommends budgeting roughly 2x the size of your indexed payload fields — the metadata you filter on, like document source, department or date — because payload indexes carry their own memory cost. Finally, Qdrant advises adding around 20% headroom on top of the final RAM and disk totals, before that infrastructure ever sees production traffic.

Milvus takes a segment-based approach: its sizing tool defaults to a 512MB segment size, but its own guidance says query nodes under 16GB of RAM should use 1GB segments instead — a direct acknowledgement that smaller nodes need different index geometry to avoid memory pressure. That's a concrete, vendor-stated threshold UK teams can use when deciding whether a lean single-node deployment is still appropriate or whether it's time to move to larger query nodes.

For teams running Postgres with pgvector rather than a dedicated vector engine, a 2026 scaling guide puts real numbers on the same problem: 5 million vectors at 1536 dimensions may need 8–16GB of working memory just during index build, and a single 1536-dimension embedding row occupies roughly 6KB. The same guide's cache-pressure warning is arguably the most operationally useful figure in this entire space: if your index exceeds 50–70% of system RAM, expect cache pressure during searches — in plain terms, your queries will slow down under load well before you run out of disk space.

Embedding Dimension Is a Cost Decision, Not Just an Accuracy Decision

It's tempting to pick the embedding model with the best retrieval accuracy and treat the infrastructure cost as a rounding error. Supabase's own comparison shows why that's a mistake: running the same document set through Postgres, a 384-dimensional embedding from all-MiniLM-L6-v2 used 4GB of RAM, while a 1536-dimensional embedding from text-embedding-ada-002 used 7.5GB — the model choice alone shifted the memory footprint by a factor that matters at any real scale.

Supabase's underlying point is that fewer dimensions mean less computation per query and let more of the dataset or index fit in memory — which is precisely the lever that determines whether you stay under that 50–70% cache-pressure threshold or blow past it. For UK teams weighing a marginal accuracy gain from a larger embedding model against a doubling of RAM requirements, this is a genuine trade-off to model explicitly, not an afterthought to discover in a production incident.

• •384-dimension embedding (all-MiniLM-L6-v2): 4GB RAM in Supabase's Postgres test
• •1536-dimension embedding (text-embedding-ada-002): 7.5GB RAM for the same corpus
• •Smaller or quantised embeddings reduce both memory footprint and per-query computation

Worked Example: Sizing the Retrieval Tier for a 5-Million-Document Corpus

Take a UK enterprise knowledge base of around 5 million document chunks, embedded at 1536 dimensions — a realistic size for a large document archive or a multi-year support-ticket history. The pgvector scaling guidance says this workload may need 8–16GB of working memory purely for the index build step, before any query traffic arrives. Each embedding row at this dimensionality runs to roughly 6KB, which means row count and dimension together — not vector count alone — set your storage bill; halving your dimension count (as the Supabase comparison shows) changes that arithmetic substantially.

The same guidance recommends planning for 2x growth in vector count as a baseline assumption, not a future upgrade project — so the retrieval-tier BOM for this corpus should be sized against roughly double the current document count from day one, with the 20% headroom Qdrant recommends layered on top of whichever engine you choose.

What this worked example deliberately does not do is attach a generation-tier GPU spec or a pounds-and-pence total, because none of the public vendor sizing documentation reviewed here publishes VRAM or GBP figures for this scenario — and inventing them would be worse than leaving the gap. The honest answer is that your retrieval-tier BOM (RAM, disk, index headroom, replication) should be quoted and priced separately from your generation-tier BOM (GPU, VRAM), which is exactly why a dedicated sizing tool for the accelerator side is worth using once your retrieval tier is specified.

Cloud vs On-Premise in the UK: Sovereignty Comes Before the Spreadsheet

For UK organisations, the cloud-versus-on-prem decision for RAG isn't purely a CapEx-versus-OpEx exercise — it's shaped first by where embeddings, payload metadata and query logs are allowed to live. RAG systems store considerably more than raw vectors: indexed payload fields, retrieval logs and often the source document snippets themselves all sit in the retrieval tier, and all of it is subject to GDPR and UK GDPR obligations around retention, access control and data residency the moment it contains personal or commercially sensitive data.

The procurement discipline this implies is to request a full-stack bill of materials in pounds — vector-store node specification, PostgreSQL or vector-engine RAM, SSD capacity for indexes plus WAL/snapshot storage, and GPU capacity for generation quoted as a separate line — rather than accepting a single blended figure. That structure makes it far easier to evaluate on-premise ownership against managed cloud services on a like-for-like basis, and to route the capital-intensive generation hardware through options such as our overview of server financing options if that suits your cash-flow position better than an upfront purchase.

Teams building their first serious on-premise deployment should also treat the retrieval-tier storage question as an infrastructure decision in its own right — index files, snapshots and payload stores need fast, resilient disk, which is why it's worth reading our breakdown of storage solutions for AI and analytics alongside any vector-database sizing exercise, and our practical walkthrough of building an on-prem AI cluster before committing to a specific architecture.

Scaling Tiers: The Thresholds Vendors Actually Publish

Rather than inventing arbitrary tier boundaries, it's more useful to anchor scale-up decisions to the thresholds vendors themselves publish. Milvus draws an explicit line at 16GB of query-node RAM: below it, use 1GB segments instead of the 512MB default, because smaller nodes need different index geometry to stay stable. That's a concrete signal that your current node class is undersized for its segment configuration, not a vague 'consider scaling soon.'

The pgvector cache-pressure warning gives a second, arguably sharper threshold: once your index exceeds 50–70% of system RAM, expect search performance to degrade under load. That's the point at which a self-hosted pgvector deployment on a general-purpose Postgres instance should be re-evaluated — either by adding RAM, reducing embedding dimensionality, or migrating to a dedicated vector engine sized with the Qdrant or Milvus formulas above.

Finally, the guidance to plan for 2x growth in vector count as standard practice means the tier transition itself should be procured ahead of the trigger point, not reactively after query latency complaints start arriving — a lesson that applies whether the underlying engine is Qdrant, Milvus or pgvector.

The Strategic Checklist: What to Actually Ask Before You Sign

Before approving a RAG infrastructure budget, UK IT leaders should be able to answer each of the following from the vendor's own numbers, not from a generic architecture diagram.

• •Is the retrieval-tier BOM (RAM, disk, replication, index headroom) quoted separately from the generation-tier GPU spec?
• •Has embedding dimension been costed as a memory decision — using something like the 4GB-versus-7.5GB comparison as a sanity check — not chosen on accuracy alone?
• •Does the sizing include Qdrant's ~20% headroom and ~2x payload-index allowance, or Milvus's segment-size rules for your node class?
• •Is the deployment planned against 2x current vector-count growth, and is there a defined RAM-to-index-size threshold (around 50–70%) that triggers a scale-up review?
• •Have data residency, retention and access-control obligations under GDPR/UK GDPR been mapped against every store in the pipeline — vectors, payload fields and query logs — not just the primary database?

Sources

Every figure in this article traces to the sources below.

• •Qdrant — capacity-planning formulas for RAM, disk, replication and payload indexes
• •Milvus — sizing tool and segment-size guidance for query nodes
• •Supabase — RAM comparison across embedding dimensions in pgvector
• •Dev.to — pgvector scaling guide: build memory, row size and cache-pressure thresholds