Qwen 2.5 72B is one of the strongest open-weight dense models for multilingual work and coding, but it is a serious piece of infrastructure to host. Because every one of its 72.7 billion parameters is active on each forward pass, its VRAM footprint sits firmly in the Llama-70B class and rarely fits on a single accelerator at usable precision. The interactive calculator below turns your context length, batch size, quantisation and precision into an exact GPU count, VRAM budget, power draw, cooling load and indicative cost.
| Precision | GPUs (H100) | VRAM | Throughput | From |
|---|---|---|---|---|
| FP16 | 4× | 220.7 GiB | 1,769 tok/s | £3,397/mo |
| FP8 | 4× | 151.7 GiB | 3,539 tok/s | £3,397/mo |
| INT4 | 2× | 117.1 GiB | 3,539 tok/s | £2,279/mo |
All figures on this page are indicative estimates for planning only and are subject to change; hardware and throughput vary with configuration and tuning, and any monthly finance figure is subject to credit approval and is not a quotation.
Why Qwen 2.5 72B needs multiple GPUs
Qwen 2.5 72B is dense, not a mixture-of-experts model, so all 72.7 billion parameters load and compute on every token. Weights alone dominate the VRAM budget, and the 80-layer stack must be held resident throughout inference. At full or half precision the model exceeds the memory of any single mainstream data-centre GPU, so a multi-GPU node with tensor parallelism is the norm. The calculator shows exactly how the split falls out for your chosen precision and accelerator.
Context length and the KV cache
The 128k-token context is a defining feature of Qwen 2.5 72B and the biggest swing factor in your hardware sizing. Grouped-query attention keeps just 8 key-value heads, which sharply trims the KV cache compared with full multi-head attention — but at long context and high concurrency that cache still grows into many gigabytes on top of the weights. If you plan to exploit the full window for RAG, long documents or agents, size for it explicitly; the calculator models cache growth directly.
Interconnect, power and cooling
Once Qwen 2.5 72B spans several GPUs, the link between them shapes throughput. NVLink-connected accelerators keep tensor-parallel traffic off the PCIe bus and deliver materially better tokens-per-second than PCIe-only pairings, which matters for interactive latency. A loaded multi-GPU node also draws real power and rejects real heat, so UK data-centre planning must account for rack power density and airflow. The calculator estimates draw and thermal load so your facility team can plan provisioning with confidence.
FAQs
Can Qwen 2.5 72B run on a single GPU?
Not comfortably at full or half precision — as a dense 72.7-billion-parameter model its weights exceed the memory of a single mainstream data-centre GPU. Aggressive 4-bit quantisation can bring it close to one very large accelerator, with some quality trade-off. For production throughput, most deployments use a multi-GPU node. Run the calculator to see the threshold for your chosen GPU and precision.
How much VRAM does the 128k context add?
Beyond the weights, the KV cache scales with context length, batch size and precision. Qwen 2.5 72B uses grouped-query attention with only 8 key-value heads, which keeps the cache far smaller than full multi-head attention would. Even so, using the full 128k window at scale adds a significant VRAM overhead. The calculator quantifies exactly how much cache your workload needs on top of the model.
Is Qwen 2.5 72B similar to Llama 70B for hardware?
Very much so. Both are dense models of roughly the same parameter count, so their weight footprint, GPU count and power envelope land in the same bracket. The main practical differences are Qwen's 128k context window and its stronger multilingual and coding performance, which can shift how much KV-cache headroom you should budget. The calculator reflects Qwen's specific configuration rather than a generic 70B estimate.