A conventional server is a sealed bargain: the GPUs, memory and NVMe you buy on day one stay inside that box until it is scrapped, whether or not they are busy. Composable disaggregated infrastructure (CDI) breaks the bargain. The expensive devices move out of the server into shared chassis on a high-speed fabric, and software attaches them to whichever host needs them, then takes them back. This guide explains how CDI works, how it differs from hyperconverged and blade systems, what it is good at, and where its limits are, using Liqid’s published specifications as the worked example.
Composable disaggregated infrastructure in one paragraph
CDI has two halves, and the name gives both. Disaggregated means the devices that usually live inside a server (GPUs, accelerators, NVMe drives, memory, DPUs) are physically separated from it and housed in their own enclosures. Composable means software decides, at any moment, which of those devices belongs to which server. A host might be composed with two GPUs in the morning, twelve in the afternoon and none overnight, without anyone opening a chassis or moving a cable.
To the server, a composed device should look like it is plugged into the motherboard. Liqid, whose LIQID Matrix software is built around the idea, states that composed GPUs appear to the host as native bare-metal PCIe devices. That is the practical difference from cloud-style virtualised GPUs: the operating system and drivers see real hardware, so most applications run unchanged.
Disaggregated and composable are not the same thing
It helps to separate the two words, because products mix them. You can disaggregate without composing: a JBOF (just a bunch of flash) shelf cabled permanently to one host moves storage out of the server, but nothing reassigns it. NVMe over Fabrics does the same for flash across Ethernet, which our guide to NVMe over Fabrics covers.
You can also compose without device-level disaggregation. Frame-based systems such as HPE Synergy pool compute, storage and fabric inside a chassis and deploy them from software templates, but the GPUs and memory still sit inside the compute modules. CDI in the Liqid sense does both: individual devices leave the server, and software composes them across standard rack servers from different vendors.
The four parts of a CDI system
Most CDI builds have four layers, and the smallest leave out the switch. In Liqid’s current generation they look like this:
- •Expansion chassis hold the devices. Liqid’s EX-5410P is a 4U box with ten slots for full-height, full-length, double-wide PCIe Gen5 cards of up to 600 W; its EX-5410C holds ten CXL memory modules, up to 20 TB.
- •A fabric switch connects chassis to hosts. Liqid’s FX-5016-2 has sixteen PCIe Gen5 x16 ports at 128 GB/s each and 115 ns port latency; the smallest bundles skip the switch and cable hosts straight to one chassis.
- •A host adapter in each server joins it to the fabric. Liqid’s HX-5160P-1C is a half-height PCIe Gen5 x16 card drawing at most 27 W, and needs no additional software on the host.
- •Composition software owns the map of which device belongs to which host. LIQID Matrix exposes that through a GUI, a CLI and a REST API, and plugs into schedulers such as Kubernetes and Slurm.
Which fabric: PCIe, CXL or Ethernet?
The fabric decides what can be disaggregated. PCIe is the natural choice for GPUs, FPGAs and NVMe, because those devices already speak PCIe; a Gen5 x16 link carries up to 128 GB/s, which is the same lane width the card would get in a motherboard slot.
CXL (Compute Express Link) runs on the PCIe physical layer but adds memory semantics, so a CPU can treat external DRAM as its own memory rather than as an I/O device. That is what makes memory pooling possible. Liqid’s CXL 2.0 kit uses its own chassis, switch and adapter, separate from the PCIe GPU fabric. Our explainer on what CXL is covers the protocol.
Ethernet and InfiniBand come in at the edges. Liqid’s 2023 Matrix datasheet lists multi-fabric support across PCIe, Ethernet and InfiniBand, and storage disaggregation over Ethernet is the job NVMe-oF already does. For devices that need full local bandwidth and latency, though, a native PCIe or CXL fabric is the closer substitute for a motherboard slot.
CDI versus fixed servers and HCI
Fixed servers, hyperconverged infrastructure (HCI) and CDI answer the same question, how to add capacity, at different grain sizes. A fixed server scales by the box: need more GPUs than it holds and you buy another server. HCI scales by the node: compute, storage and virtualisation grow together, which suits general virtualisation (size it with our HCI node sizing calculator). CDI scales by the device: add one GPU or one memory module to the pool and give it to the host that needs it.
The grain size matters most for the scarcest, most expensive parts. Liqid’s own L40S reference design notes that traditional servers are typically limited to 4 to 8 GPUs; its UltraStack bundles compose up to 30 to a single host. And because devices return to the pool, a GPU bought for one team’s project is not stranded when the project ends.
View the data behind this chart
| Fixed server | HCI | Composable | |
|---|---|---|---|
| Unit of scaling | Whole server | Whole node | One device |
| GPUs per host | Chassis slots | Node slots | Up to fabric pool |
| Idle device | Stranded | Stranded in node | Back to pool |
| CPU refresh | Replace all | Replace node | Host only |
| Best for | Steady loads | Virtualisation | Uneven GPU use |
Where CDI is being used
The use cases cluster around resources that are expensive and unevenly used. GPU pooling serves AI inference, VDI, rendering and research groups whose demand rises and falls; Liqid sells validated SmartStack bundles that share 10 to 30 GPUs across 4 to 16 hosts. Many-model inference uses the opposite shape: Liqid’s UltraStack with AMD Instinct MI350P puts 4 to 30 GPUs on one host, one model per GPU. CXL memory pooling targets vector and graph databases, RAG and KV cache, where workloads run out of memory before compute.
In research, Liqid lists Durham University’s Institute for Computational Cosmology, which runs the COSMA system for the UK’s DiRAC facility, among its composability case studies. Micron’s Abaco rack, built on Liqid’s fabric with up to 160 TB of pooled DDR5 (its memory chassis use 4 TB CXL cards, 40 TB each), has its first deployment at the US Pacific Northwest National Laboratory.
The trade-offs
CDI is not free. Each composed device sits behind several PCIe switch stages (Liqid rates each at about 115 ns, and its Abaco brief puts the chassis path at roughly 500 ns), and every device composed to a host shares that host’s fabric link: in SmartStack bundles, one Gen5 x16 adapter at up to 128 GB/s. Liqid supports direct GPU-to-GPU traffic inside the fabric, which keeps peer traffic off that link, but workloads that stream heavily between host memory and many GPUs need sizing with this in mind.
Only PCIe devices can be pooled this way. SXM-module GPUs on NVLink baseboards are not PCIe cards, so for one large, tightly coupled training job an HGX or NVIDIA DGX system remains the purpose-built tool. There are practical limits too: servers and GPUs should be on, or be checked against, the vendor’s compatibility list, some with riser or BIOS conditions, and Liqid’s EX-5410P GPU chassis is specified for 0 to 27 °C, a lower ceiling than most servers.
How to tell if CDI fits your estate
Start with utilisation, not product sheets. If some GPU servers queue work while others sit idle, if a single job keeps outgrowing the biggest server you can buy, or if memory is the reason you over-specify every host, CDI has something to recover. If every GPU is busy where it sits, it does not.
Then check the hardware you own against the vendor’s compatibility list; our Liqid compatibility summary breaks Liqid’s list down by vendor. Finally, model the pooled estate against the fixed one with your own demand numbers before buying. Vendor efficiency figures are best cases, and Liqid itself labels some of them targets rather than measurements.