Fine-Tuning vs RAG: The UK Infrastructure Decision for 2026

The hardware fork: what RAG and fine-tuning actually commit you to

Every fine-tuning vs RAG comparison online debates hallucination rates and accuracy. Almost none tells you what each path does to your procurement list. That's the gap worth closing, because the fork is real and it's a hardware fork: choose fine-tuning and you're buying or renting training-capable GPU memory; choose RAG and you're building a retrieval stack that runs on standard serving hardware.

The numbers make the split concrete. Full fine-tuning of a 70B model needs around 700GB of VRAM according to 2026 practitioner data from GigaGPU — territory that puts you firmly in multi-GPU cluster deployment. QLoRA, the parameter-efficient alternative, brings that same 70B model class down to about 46GB VRAM at 4-bit precision, per Modal's published VRAM tables (from 2024). That's the gap between an eight-GPU pod and a single high-end data-centre card. RAG sidesteps this arithmetic entirely: it adds an embedding model, a vector index, a retrieval service and an LLM serving tier — none of which need gradient-capable memory.

When RAG wins: freshness, retrieval, and UK data control

RAG earns its keep when the underlying knowledge changes faster than a retraining cycle can keep up — policy documents, pricing tables, regulatory guidance, internal wikis. Because the model's weights are never touched, updating the answer set is a reindexing job, not a training job. That matters in UK-regulated sectors where source documents get revised routinely and auditors want to trace exactly which document backed a given answer. See our detailed RAG infrastructure requirements for the UK-specific build-out.

It also matters for data residency. With RAG, the sensitive corpus — customer records, case files, contract text — can sit in a UK-hosted vector database and never leaves that environment for training; the LLM only ever sees retrieved snippets at inference time. That's a simpler data-protection story than fine-tuning, where training data effectively becomes part of the model weights, and those weights then need hosting, backup and audit as if they were the data itself.

When fine-tuning wins: behaviour, tone, and training-capable GPUs

Fine-tuning solves a different problem: not 'what does the model know' but 'how does the model behave.' If you need consistent tone of voice, domain-specific phrasing, a house style for customer communications, or a narrow task performed the same way every time, adapting the weights — even a small slice of them via LoRA or QLoRA — tends to be more reliable than stuffing instructions into every prompt.

This is where the hardware conversation becomes unavoidable. Modal's VRAM tables (published in 2024) show QLoRA fine-tuning a 7B model needs about 9GB at 8-bit or about 5GB at 4-bit, while standard LoRA at 16-bit needs roughly 15GB for the same model class. Move to 13B and LoRA at 16-bit needs about 28GB, versus roughly 9GB for QLoRA at 4-bit. At 70B, QLoRA at 4-bit needs about 46GB against roughly 146GB for LoRA at 16-bit. Every one of those figures is a GPU-buying decision, not a config toggle — and DevTechTools' analysis (published in late 2025) notes QLoRA roughly halves or better the VRAM burden of standard LoRA in comparable consumer-GPU scenarios, which is exactly why QLoRA has become the default entry point for UK teams without a dedicated training cluster.

Hybrid architecture: building both without duplicating your stack

Most UK production deployments need both freshness and consistent behaviour, which is why hybrid architectures — a fine-tuned adapter layered on top of a RAG pipeline — are increasingly the pragmatic default rather than a compromise. The pattern: a base model gets a small LoRA or QLoRA adapter trained once on your house style, tone and domain vocabulary; that adapted model then sits behind a retrieval layer that supplies current facts at inference time. You get consistent voice and a modest training footprint, plus a knowledge base you can refresh without retraining anything.

For UK compliance, this split is useful because it separates two different data problems. The adapter is trained once on a curated, reviewed style dataset — small enough to audit, with a training footprint governed by the VRAM figures above. The retrieval corpus, which holds the sensitive, frequently changing material, stays in a UK-hosted vector store and is never used to update model weights. That's a considerably cleaner answer for a data protection officer than 'the customer records are somewhere inside the fine-tuned weights.' If you're building this from scratch, the sequencing lessons in building an on-prem AI cluster apply directly to the adapter-training half of the stack.

TCO in the UK: a worked example

Take a mid-sized UK financial services firm building an internal assistant for underwriters — it needs to answer questions against constantly updated policy wordings and also respond in a consistent, compliance-approved tone. Sizing the model at 13B parameters, a QLoRA fine-tune fits on a 24GB GPU such as an RTX 3090 or RTX 4090, per a 2026 practitioner write-up — a workstation spec that sits comfortably inside the AI workstation pricing Workstation.co.uk lists starting around £6,000 for 2026 UK builds. That's the one-off adapter-training cost.

The retrieval side of the same project doesn't need that GPU at all — it needs a vector database, an embedding pipeline and a serving tier, none of which require training-capable hardware. If the firm instead colocates a GPU cluster for a heavier fine-tuning workload, Armason's 2026 UK benchmarks put a 5–15kW rack at roughly £180 to £260 per kW per month, rising to £300–£420 per kW per month for 50kW-plus liquid-cooled deployments. Those figures only apply once you've committed to the training-capable path. The practical read: the fine-tuning branch of this project carries a hardware bill with a name and a number attached to it; the RAG branch carries a subscription-and-storage bill that scales with data volume and query traffic instead. Compare both against renting the compute — our self-hosting LLM vs cloud GPU cost breakdown covers that trade-off in more depth.

Latency, throughput and the long-term operational bill

Accuracy debates aside, the architectural choice also shapes runtime performance. RAG's generation step is preceded by a retrieval hop — an embedding model, a vector index lookup, and often a reranking pass — before the LLM serving tier ever sees the prompt; a fine-tuned model, by contrast, generates directly from the adapted weights with no retrieval step in between. That extra hop is the operational cost of freshness: every RAG query pays a search-and-rerank latency tax that a fine-tuned model doesn't carry.

The long-term overhead differs just as much as the day-one hardware bill. RAG's ongoing cost sits in the data pipeline: keeping the vector index current as source documents change, re-embedding updated content, and monitoring retrieval quality and latency as the corpus grows — none of which touches the GPU fleet, but all of which needs sustained data-engineering attention and governance oversight. Fine-tuning's ongoing cost sits in retraining and versioning: any meaningful change in desired behaviour means another training run, which re-triggers the same VRAM and GPU procurement questions covered above, plus the need to track which adapter version is live and roll back cleanly if a retrain regresses quality. Budgeting for either path means budgeting for that maintenance loop, not just the first build.

Hardware and cloud compute for LoRA/QLoRA: what to actually buy

Map the workload to the GPU rather than guessing. Use the AI GPU requirements calculator to translate model size and adapter method into a concrete VRAM target before you price anything.

• •7B models: QLoRA needs roughly 8–12GB VRAM per a 2026 practitioner write-up, with an RTX 3060 12GB cited as sufficient; Spheron's 2026 estimate of 6–10GB with a 4-bit quantised base points the same way — consumer GPUs with 12GB or more are workable.
• •13B models: QLoRA trains on a 24GB card such as an RTX 3090 or RTX 4090, per the same 2026 write-up.
• •Scaling up: Introl's 2026 guide puts QLoRA minimums at RTX 3090 24GB or RTX 4080 16GB, recommending A100 40GB or L40S 48GB once the workload outgrows those cards.
• •70B models: QLoRA at 4-bit needs about 46GB VRAM per Modal (2024 data), or about 52GB per GigaGPU's 2026 estimate specifically for Llama 3 70B — either way this is multi-GPU or data-centre-card territory, not a desktop build.
• •UK pricing tiers: Workstation.co.uk's 2026 listings put sub-£30,000 builds against the 7B–13B QLoRA range, with the £30,000–£150,000+ bracket covering 70B-class and multi-GPU training builds — see our GPU accelerators guide for card-level detail.

Data governance and GDPR: where each approach creates risk

Under UK GDPR, the two approaches raise different residency questions. Fine-tuning bakes training data into model weights — if that training happened on a third-party cloud platform outside the UK, a data protection officer needs to trace exactly what data went into the weights and where the training run physically executed. RAG keeps the sensitive corpus in a separate, addressable store, which is easier to keep within a UK data centre and easier to redact, delete or update on a subject-access request, because the LLM itself never 'memorised' the record.

In practice, 2026 UK best practice treats the retrieval corpus as the primary regulated asset: encrypt it at rest, log every retrieval query for audit, and keep the vector database in the same jurisdiction as the source documents. For fine-tuning, the equivalent discipline is stricter curation of training data before it ever reaches a GPU — because once weights are trained, removing a single record's influence isn't a deletion operation, it's a retraining operation.

Decision framework for UK IT leaders

A serviceable framework: if the knowledge changes more often than you can retrain, or the source data is sensitive and needs to stay auditable and deletable, start with RAG. If the requirement is about voice, format or a narrowly defined task rather than knowledge, and you already have — or can budget for — a training-capable GPU, LoRA/QLoRA fine-tuning is the more direct fix. If both are true, and for most UK enterprise use cases they eventually are, build the hybrid: stand up retrieval first because it de-risks the project without committing to GPU procurement, then layer in a lightweight adapter once you know exactly which behaviours need fixing.

Skills matter as much as hardware. RAG projects lean on search, data engineering and DevOps skills most UK IT teams already have in some form. Fine-tuning projects need ML engineering literacy — optimizer state, batch sizing, quantisation trade-offs — a rarer skillset outside dedicated data science teams, which is why many UK mid-market firms outsource the adapter-training step to a specialist while keeping the retrieval stack in-house. Decide it as a hardware and governance commitment first, and an accuracy debate second.

Sources

Every figure in this article traces to the sources below.

• •Modal — QLoRA/LoRA VRAM tables by model size and precision
• •Morgann Riu — 2026 practitioner GPU guidance for LoRA/QLoRA
• •Introl — 2026 fine-tuning infrastructure scale guide
• •Spheron — 2026 GPU VRAM requirements breakdown
• •GigaGPU — 2026 LoRA vs QLoRA vs full fine-tuning GPU comparison
• •Workstation.co.uk — 2026 UK AI workstation pricing
• •Armason — 2026 UK GPU colocation pricing benchmarks
• •DevTechTools — 2026 LoRA vs QLoRA memory efficiency analysis