Llama 3.1 8B is Meta's compact 8-billion-parameter dense model, and its appeal is straightforward: it delivers genuinely useful reasoning and retrieval from a single mainstream GPU. For UK teams weighing cost against capability, it is often the sweet spot for high-throughput serving, edge deployment and RAG. The interactive calculator on this page turns your context length, batch size and precision choices into precise hardware figures.
| Precision | GPUs (L40S) | VRAM | Throughput | From |
|---|---|---|---|---|
| FP16 | 2× | 48.9 GiB | 2,066 tok/s | £839/mo |
| FP8 | 1× | 41.3 GiB | 2,066 tok/s | £667/mo |
| INT4 | 1× | 37.5 GiB | 4,132 tok/s | £667/mo |
All figures on this page are indicative estimates for planning only and are subject to change; hardware and throughput vary with configuration and tuning, and any monthly finance figure is subject to credit approval and is not a quotation.
How much VRAM does Llama 3 8B need?
At full 16-bit precision the weights alone sit comfortably inside a single 24GB card, leaving room for the KV cache and activations. Because Llama 3.1 8B uses grouped-query attention with only eight key-value heads, its KV cache stays lean even as you stretch towards the 128k context window — a real advantage over models with wider attention. The calculator adds your context length, concurrent requests and precision together so you see the true working-set VRAM, not just the weight footprint.
How many GPUs to serve it?
For most workloads Llama 3.1 8B is a single-GPU model — the whole point of an 8-billion-parameter dense architecture is that it avoids the multi-card sharding and NVLink fabric that larger models demand. You scale out for throughput and redundancy rather than because one model instance won't fit. That keeps deployment simple, latency low and interconnect cost near zero. The calculator shows exactly when a second card earns its place for your concurrency target.
Quantisation: run it on modest hardware
INT4 quantisation roughly quarters the weight footprint, letting Llama 3.1 8B run on entry-level accelerators, workstation cards and edge boxes with very little quality loss for most tasks. This is what makes it a favourite for cost-sensitive serving, on-premise RAG and branch deployments across the UK. FP16 or INT8 preserves maximum fidelity for evaluation and fine-tuning. Toggle precision in the calculator to watch VRAM, power draw and card requirements shift in real time.
FAQs
Can Llama 3 8B run on a single GPU?
Yes. Llama 3.1 8B is a dense 8-billion-parameter model that fits comfortably on one modern 24GB-class GPU at 16-bit precision, and on far more modest hardware once quantised to INT4. Unlike larger models, it needs no multi-GPU sharding or NVLink fabric. Add cards only when you want more throughput or redundancy, not to fit the model.
How much VRAM does the 128k context use?
The full 128k context grows the KV cache, but Llama 3.1 8B's grouped-query attention — just eight key-value heads — keeps that growth modest compared with wider-attention models. Long-context serving is very feasible on a single card. Enter your target context length and concurrency in the calculator to see the exact KV cache and total VRAM for your setup.
Is Llama 3 8B good for RAG and edge deployment?
It is one of the strongest fits. Its small size, low VRAM footprint and efficient GQA make it fast and cheap to serve, so it suits high-throughput RAG pipelines, retrieval assistants and edge or branch hardware where power and cooling are constrained. INT4 quantisation extends this further, letting you run it economically close to where your data lives.