Software-Defined Storage Explained: SDS Architecture and Costs

What Is Software-Defined Storage and How Does It Function?

Traditional enterprise storage relies on monolithic storage area network (SAN) or network-attached storage (NAS) appliances. These systems bundle proprietary dual-controller hardware, specialised backplanes, custom operating systems, and vendor-qualified hard drives or solid-state media into a single chassis. When an organisation requires more capacity or compute performance, it must purchase vendor-approved drive expansion trays or execute a disruptive, costly controller refresh.

Software-defined storage (SDS) breaks this rigid coupling by moving the entire storage intelligence—including data placement, health monitoring, volume aggregation, tiering, and protection—into an independent software layer. This software layer pools the internal storage drives of standard x86 servers over standard IP or Ethernet network links. Rather than trusting resilience to a hardware RAID controller or dual-engine SAN chassis, the SDS software manages failure domains across individual disks, server nodes, or entire datacentre racks.

In most software-defined architectures, incoming data writes are coordinated by a centralised or distributed control plane that fragments, replicates, or encodes blocks, files, or objects according to predefined policy. The underlying servers typically do not require proprietary ASICs or specialised fabric cards; standard disks and network interfaces normally provide the physical capacity, though some platforms can optionally use specialised hardware, while the software handles health states, scrub routines, and automatic cluster healing.

SDS vs SAN vs NAS: Architecture and Operational Trade-Offs

Comparing SDS with legacy storage architectures requires evaluating control placement, failure handling, and scalability paths. In a traditional SAN or NAS, failure resilience is hardware-centric. Dual active-active or active-passive controllers commonly share an internal midplane such as SCSI or SAS, though modern arrays may use other interconnects including PCIe or NVMe-based fabrics. If a physical controller fails, the secondary engine takes over the disk cache and keeps data paths open. However, if the backplane or chassis suffers catastrophic damage, the array becomes a single point of failure.

By contrast, SDS treats entire server nodes as independently replaceable units within a wide cluster, designing for node failure without losing data or service availability. If an entire server drops offline due to a blown power supply or mainboard failure, the remaining nodes detect the dropped heartbeat and dynamically reconstruct missing blocks across the surviving hardware. Scalability also differs fundamentally: traditional SAN systems often scale up until controller CPU or backplane bandwidth becomes a constraint, at which point organisations may face a costly controller upgrade or migration to a different platform. SDS scales out incrementally by adding standard rack-mount servers to an active pool.

However, SDS introduces new operational demands. While a SAN conceals controller complexity behind proprietary management firmware, an SDS cluster demands rigorous attention to networking bandwidth, east-west latency, and node-level compute consumption. The storage intelligence directly draws CPU cycles and memory from the host servers, meaning system administrators must carefully plan cluster overhead rather than outsourcing performance management to a vendor black box.

• •Failure domains: SAN architectures protect against component faults within a single chassis; SDS protects across nodes, racks, and discrete server chassis.
• •Scalability mechanics: SANs scale up vertically until controller throughput limits are hit; SDS scales out horizontally by appending standard server nodes.
• •Operational load: SAN arrays offer turn-key appliance management; SDS demands deliberate monitoring of network interfaces, memory pools, and host processor utilisation.

Data Protection Mechanics: 3x Replication vs Erasure Coding

The primary design decision governing software-defined storage capacity and performance is the data protection policy. Storage architects typically choose between two mechanisms: multi-copy replication and algorithmic erasure coding (EC).

A standard 3x replication policy writes three identical copies of every data block to three separate failure domains, such as three distinct server nodes. This provides simple recovery: if a disk or server fails, the cluster restores redundancy by reading directly from a surviving replica and copying it to an available drive. The operational cost of this simplicity is immense. A 3x replication policy imposes a 200% raw-capacity overhead, which means 300 TB of physical disk is consumed to store 100 TB of usable enterprise data.

Erasure coding splits incoming data into 'k' data fragments and computes 'm' parity fragments, distributing all pieces across distinct nodes. A common 4+2 profile takes 4 data chunks, generates 2 parity chunks, and writes 6 fragments in total. This profile tolerates the concurrent loss of any 2 nodes while reducing raw-capacity overhead to exactly 50% (a 1.5x raw-to-usable multiplier). A 6+3 Reed-Solomon profile similarly carries a 50% space overhead while tolerating 3 simultaneous faults, though recommended minimum node counts for 6+3 vary between implementations. In Ceph deployments, erasure-coded layouts are commonly presented as 1.33x to 1.5x raw capacity for usable data in typical profiles such as 6+2, 8+3, or 4+2, compared with 3x for replication.

For dense, capacity-focused clusters with high node counts, erasure coding yields dramatic efficiency gains. In one widely documented implementation, a 5+2 profile incurs a 40% capacity overhead across a minimum of 7 nodes, while a 7+2 layout reduces overhead to 29% across at least 9 nodes; other platforms may use different minimum node recommendations. In hyperscale or deep-archive tiers, an aggressive 17+3 profile can drop raw overhead to about 18% in documented implementations, though recommended node counts and suitability differ between vendors. To understand how these distribution methods apply across different access patterns, you can understand block, file, and object storage.

The Hidden Taxes: Network and Processor Penalties in SDS

While erasure coding slashes storage hardware acquisition costs, it is not an unalloyed win. Research into distributed storage systems confirms that erasure coding introduces measurable penalties in compute load, east-west network saturation, and write latency. In Ceph-style distributed environments, these trade-offs dictate exactly where specific profiles should be deployed.

The critical friction point occurs during the parity-update and rebuild path. With 3x replication, an incoming write is copied directly to destination drives; a rebuild merely streams surviving blocks over the network without algorithmic computation. With erasure coding, many writes—especially random small-block modifications—require the host or storage daemon to read existing fragments, calculate parity, and distribute fragments across multiple nodes, increasing CPU and network load.

When a disk or node fails in an erasure-coded cluster, the rebuild process cannot simply copy surviving data. The cluster must read multiple fragments from healthy nodes across the datacentre network, compute parity equations in CPU memory to regenerate missing fragments, and write the newly calculated data to target disks. This consumes significant CPU cycles and floods inter-node switches with east-west reconstruction traffic. Operational guidance from Ceph and other SDS platforms typically reserves erasure coding for cold archives, backups, and capacity-heavy object tiers, while replicated pools are preferred for latency-sensitive, small-write-intensive transactional workloads.

Hardware Economics and the UK Deployment Reality

Evaluating software-defined storage in the UK requires grounding decisions in total infrastructure economics. With raw mechanical storage sitting at the previously noted £12.90 per TB baseline, raw media acquisition is often less significant than resilience overhead, licensing, and server infrastructure, especially at larger scales.

Because raw media is relatively affordable but still a significant cost at scale, running a 200% capacity penalty on 3x replication can become financially inefficient as datasets grow into the hundreds of terabytes and beyond. However, shifting to low-overhead erasure coding demands higher initial server node counts. A 5+2 profile requires at least 7 physical nodes to establish proper failure domains, and a 7+2 profile requires a minimum of 9 nodes. For a smaller business, purchasing 7 to 9 enterprise servers, populated with network cards and power supplies, creates a capital hurdle that can exceed the cost of a basic dual-controller array.

At the lower end of the market, entry-level hardware packages demonstrate how accessible basic clustering has become. For example, a complete 3-node Ceph, ZFS, or TrueNAS storage cluster built from secondary Dell EMC R740 servers was tracked on the UK secondary market at £4,985.00—an illustrative single listing demonstrating entry-level hardware outlay, though actual capital costs fluctuate depending on CPU specification, memory density, and warranty coverage. Alternatively, organisations seeking cloud-hosted object storage face recurring operational fees; a comparable hosted MinIO profile is listed around $999 per month (approximately £780 to £820 per month subject to foreign exchange volatility, or roughly £1.10 per hour) for an instance with 500 Mbps bandwidth. The economic break-even point for on-premises SDS arrives when ongoing array support fees, proprietary drive markup, and cloud egress charges surpass the operational and infrastructure cost of running an in-house multi-node cluster.

Beyond server chassis and drives, software licensing and vendor support constitute one of the largest ongoing expenses in SDS deployments. Commercial solutions like VMware vSAN (licensed on per-core subscription models), Nutanix, and DataCore SANsymphony introduce recurring annual capacity or compute fees that can match or exceed server capital costs over a multi-year refresh cycle. In contrast, open-source solutions like Ceph or TrueNAS CORE incur zero initial software licensing fees, but enterprise UK deployments typically require commercial support agreements—such as Red Hat Ceph Storage or Canonical Ubuntu Advantage subscriptions—or internal senior engineering capacity to manage production cluster operations, lifecycle updates, and node recovery.

UK Compliance, Security, and Governance Frameworks

Deploying software-defined storage across enterprise or public-sector environments in the UK introduces specific regulatory and architectural requirements. Under the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018, organisations are required to implement appropriate technical and organisational measures to ensure data confidentiality, integrity, and availability.

In an SDS deployment, data protection is inseparable from cluster architecture. Because erasure coding and distributed replication scatter fragments across arbitrary server nodes, organisations must define strict failure-domain topologies. If data handling requires strict geographic confinement or physical segmentation, the software control plane must be configured with location-aware rules that prevent data fragments from traversing unapproved network segments or resting on non-compliant nodes.

For UK public-sector workloads, infrastructure buyers must adhere to the National Cyber Security Centre (NCSC) cloud security approach. Systems managing official information must implement robust baseline security controls, including encryption at rest, secure inter-node transport encryption, role-based identity access, and rigorous supply chain management. When evaluating an SDS deployment, UK engineering teams cannot look at £/TB alone; they must ensure the software platform supports cryptographically enforced data isolation and comprehensive audit logging across all physical server endpoints.

Worked Example: Mid-Sized Enterprise Architecture Decision

Consider a mid-sized UK engineering firm holding 200 TB of usable secondary data, including simulation archives and historical project files. The infrastructure team must evaluate whether to build a 3-node replicated cluster, deploy a larger erasure-coded SDS architecture, or purchase an entry SAN.

If the team implements a 3-node SDS cluster using 3x replication, providing resilience against two drive or node failures, the storage overhead is 200%. Storing 200 TB of usable files requires 600 TB of raw disk capacity. At that £12.90/TB disk baseline, raw media costs £7,740.00. Sourcing three refurbished 2U enterprise nodes typically adds £5,000 to £6,500 in capital outlay depending on specification. When factoring in enterprise software licensing or vendor support subscriptions (often £2,500 to £4,000 annually for commercial SDS or enterprise Ceph support), the first-year hardware and software investment sits around £15,000 to £18,000, or roughly £13,000 if run entirely on community-supported open source.

Conversely, if the team deploys a 4+2 erasure-coding profile, the raw-capacity overhead drops to 50%. The firm needs only 300 TB of raw capacity to yield 200 TB usable, cutting the drive requirement to £3,870.00. However, this 4+2 layout requires a minimum of 6 physical nodes to isolate fragments across unique failure domains. The business saves £3,870 on raw storage media, but must purchase, power, and license three additional x86 server nodes—adding £5,000 to £7,000 in server capital, extra switch ports, and potential per-node or per-core software licensing fees. For clusters under roughly a few hundred terabytes, server hardware, power, and per-node licensing overheads frequently cancel out disk savings; at petabyte scale, however, the capacity savings from erasure coding comfortably surpass node hardware and licensing differentials.

Implementation Checklist and Migration Best Practices

Transitioning from legacy SAN or NAS infrastructure to software-defined storage requires structured execution across network design, hardware sizing, and data migration paths.

First, validate the network fabric. Software-defined clusters rely heavily on low-latency, non-blocking east-west networking for synchronous replication, scrubbing, and background parity rebuilds. Deploying SDS on an oversubscribed 1GbE or shared 10GbE network will bottleneck storage performance and extend rebuild durations dangerously during node outages. Dedicated storage VLANs or isolated 25GbE/100GbE switching fabrics are strongly recommended for production scale, especially for clusters with high IO rates or tight rebuild SLAs.

Second, profile application IO patterns before assigning storage policies. Do not place latency-sensitive, high-frequency transactional databases onto deep erasure-coded pools such as 6+3 or 7+2. Dedicate high-speed replicated NVMe/SSD pools to write-heavy transactional workloads, while routing bulk unstructured data, archives, and backup targets to high-density erasure-coded spinning disk pools. To compare how scale-out clusters handle these mixed workloads versus traditional arrays, compare scale-out and scale-up storage architectures.

Sources

Every figure in this article traces to the sources below.

• •SNIA — Erasure coding pros and cons in datacenter infrastructure
• •Scality — Replication vs erasure coding capacity overhead
• •OpenMetal — Ceph storage efficiency and erasure coding workloads
• •DIVA Portal — Space overhead in Reed-Solomon distributed storage profiles
• •Acronis — Storage redundancy overhead and minimum node sizing
• •disks.co.uk — UK storage price index tracking disk hardware costs
• •NCSC — UK public sector cloud security principles and data controls
• •ICO — UK GDPR and Data Protection Act 2018 security guidance
• •StackHero — UK hosted MinIO pricing metrics
• •eBay UK — Secondary market 3-node storage cluster listings