Industry update — Microsoft is bringing a native inbox NVMe-oF initiator to Windows Server, with support for both NVMe/TCP and NVMe/RDMA (RoCE/iWARP) transports now available for preview in Insider builds (Microsoft Community Hub, March 2026).
NVMe made local flash astonishingly fast by talking to a drive over PCIe with almost no protocol overhead. NVMe over Fabrics (NVMe-oF) takes that same command set and carries it across a network, so a host can reach NVMe storage that lives in another box, another rack, or another part of the fabric — while keeping latency close to a directly attached drive. This article explains what NVMe-oF actually is, the transports it runs on, why an FPGA is a natural place to implement a fabric endpoint, and where a dedicated NVMe SAN IP core fits into that picture.
The short answer
NVMe over Fabrics extends the NVMe command set beyond a single PCIe link so a host (the initiator) can access NVMe namespaces on a remote target across a network. It runs over three transport families — RDMA (RoCE, iWARP, InfiniBand), plain TCP, and Fibre Channel — and is designed to add only a small latency penalty over local NVMe. On an FPGA, the appeal is a CPU-less, deterministic fabric endpoint: the device becomes an initiator or a target in hardware. An NVMe SAN IP core is exactly that endpoint, letting an FPGA storage or recorder node join an NVMe-oF fabric rather than owning a drive only over its own local PCIe.
What is NVMe over Fabrics?
NVMe over Fabrics (NVMe-oF) is a networked extension of the NVMe storage protocol: it carries the same commands, queues and completions across a network, so a host can read and write remote NVMe storage as if it were a local drive. The transport bindings and the target/initiator model are defined in the NVMe over Fabrics specification, with vendor-neutral primers maintained by SNIA.
NVMe began as a protocol for talking to solid-state storage over PCIe. It replaced the assumptions of older, disk-era interfaces with a lean, parallel model built around many deep command queues, as described in the NVM Express specification. That model is the reason local NVMe SSDs feel so fast.
The limitation is reach. A native NVMe drive is bound to the PCIe link of the machine it plugs into. If another server — or an FPGA node on the other side of a switch — needs that storage, plain PCIe NVMe cannot get it there. NVMe over Fabrics solves this by mapping the same commands, queues and completions onto a network transport instead of a local PCIe bus. The command set the host issues is essentially the one it would issue to a local drive; what changes is the path those commands travel.
Two roles define an NVMe-oF deployment:
- The target is the side that owns the storage. It exposes one or more namespaces — the addressable storage regions of an NVMe device — onto the fabric and services the commands that arrive.
- The initiator (host) is the side that consumes the storage. It discovers targets on the fabric, connects to their namespaces, and issues reads and writes as if the storage were local.
Because the namespace abstraction is preserved end to end, software above the initiator often does not need to know whether a namespace is a local PCIe drive or a remote one reached across the fabric. That transparency is much of what makes NVMe-oF attractive.
Why decouple compute from storage?
The architectural payoff of NVMe-oF is disaggregation — separating where data is processed from where it is stored. When storage lives behind a fabric rather than inside each machine, several things become possible:
- Independent scaling. Add capacity to a shared pool without touching the compute nodes, or add compute without stranding idle drives inside every server.
- Shared, pooled flash. Multiple hosts can draw from a common set of NVMe namespaces instead of each carrying captive drives that sit half-empty.
- Placement freedom. A capture or processing node can stay compact while its storage sits elsewhere in the rack or system, connected only by a high-speed link.
For an embedded or instrumentation designer, the same idea applies at a smaller scale: an FPGA doing acquisition does not have to physically host the drives it writes to. It can be an NVMe-oF endpoint that streams into a storage node across the fabric, keeping the acquisition board lean and the storage where it is easiest to manage and cool.
The transports: RDMA, TCP and Fibre Channel
NVMe-oF does not define a single wire protocol. It defines a binding onto several transport families, each with a different trade-off between latency, hardware requirements and how well it reuses existing infrastructure.
RDMA transports (RoCE, iWARP, InfiniBand)
Remote Direct Memory Access lets one machine read and write another machine’s memory with minimal CPU involvement, which is a natural fit for NVMe’s queue model. RDMA appears in three forms: RoCE (RDMA over Converged Ethernet), which runs on Ethernet and generally expects a lossless, well-configured network; iWARP, which carries RDMA over routable TCP and is more tolerant of ordinary networks; and InfiniBand, a purpose-built HPC fabric. RDMA transports typically deliver the lowest added latency, at the cost of needing RDMA-capable network hardware.
NVMe/TCP
NVMe/TCP carries the protocol over standard TCP/IP. Its great advantage is reach and simplicity: it runs on ordinary Ethernet with no special adapters, over routed networks, using infrastructure that already exists everywhere. The trade is a somewhat higher latency and more per-packet processing than RDMA, though for many workloads that gap is acceptable given how much easier it is to deploy.
NVMe/FC
NVMe over Fibre Channel maps NVMe onto an existing Fibre Channel SAN. For organisations that already run FC, it lets NVMe traffic ride the storage network they have already invested in and know how to operate, rather than standing up a new Ethernet storage fabric.
Choosing a transport is an architecture decision, not just a networking one — it shapes latency, cost and how the FPGA endpoint is built. A short design conversation early saves a costly rework later.
Transport comparison
| Transport | Underlying network | Key characteristic | Typical fit |
|---|---|---|---|
| NVMe/RDMA — RoCE | Ethernet (often lossless / configured) | Very low added latency | Data-centre storage fabrics with RDMA NICs |
| NVMe/RDMA — iWARP | Ethernet over routable TCP | RDMA that tolerates ordinary networks | Where lossless Ethernet is not guaranteed |
| NVMe/RDMA — InfiniBand | InfiniBand fabric | HPC-grade latency and bandwidth | High-performance computing clusters |
| NVMe/TCP | Standard TCP/IP Ethernet | Runs on any existing network, no special NICs | Broad reach and operational simplicity |
| NVMe/FC | Fibre Channel SAN | Reuses established FC infrastructure | Enterprises with an existing FC storage network |
Where the FPGA fits
An FPGA is a strong home for an NVMe-oF endpoint for the same reasons it suits local NVMe: it can move data along a deterministic, CPU-less path. There is no operating system scheduling the transfers and no software stack adding jitter, so timing stays predictable even under sustained load.
There are two natural roles for the FPGA on the fabric:
- As a target, the FPGA exposes storage — local NVMe drives or another data source — onto the fabric and offloads the transport and NVMe-oF protocol work into hardware. This frees any attached host CPU from that burden and keeps the storage path deterministic.
- As an initiator, the FPGA consumes remote storage. An acquisition or processing pipeline can write across the fabric to a namespace that physically lives elsewhere, without a processor mediating each transfer.
In both roles the FPGA handles the demanding, latency-sensitive parts — queue management, data movement, transport framing — in fabric logic. This is the same division of labour that makes a hardware NVMe Host IP core attractive for local drives, extended out onto a network. It is also why an FPGA data recorder can grow beyond the drives bolted to its own board: with a fabric endpoint, its storage is wherever the fabric reaches.
Where the LDS NVMe SAN IP fits
Logic Design Solutions builds soft-IP cores that put these storage protocols directly into FPGA fabric. The one that maps onto NVMe-oF is the NVMe SAN IP line — the endpoint that lets an FPGA act as a node on a storage-area network rather than a device that owns drives only over its own local PCIe.
Conceptually, it sits at the target/initiator boundary of an NVMe-oF topology. Where the NVMe Host IP core connects the FPGA to a directly attached SSD, the SAN IP core extends that reach across the fabric — so the FPGA can present its storage onto the network, or reach storage that lives elsewhere on it. For capture-heavy systems, this pairs with the recorder-oriented cores such as the NVMe Host Recorder IP: acquisition happens on one node, and the data lands on storage the fabric makes available, all without a CPU in the data path. For a closer look at that core on its own — how it models a storage-area network inside FPGA fabric — see our explainer on what NVMe SAN IP is.
The value is architectural rather than a single headline number. By implementing the fabric endpoint in hardware, the design keeps the deterministic, offloaded character that makes FPGA storage attractive in the first place, while gaining the placement freedom of a networked storage model. Whether NVMe/TCP, an RDMA transport or Fibre Channel is the right binding depends on your latency budget and existing network — a decision worth settling early, because it shapes how the endpoint is built.
Logic Design Solutions has developed FPGA storage and interface IP since 1994, across AMD (Xilinx) and Altera devices, so the SAN IP endpoint slots alongside the wider NVMe, SATA, ARINC 429 and Ethernet line rather than standing alone.
When NVMe-oF is not the answer
NVMe-oF is powerful, but it is not free complexity. Reach for a simpler design when:
- The storage can be local. If the drives can sit on the FPGA’s own PCIe link, a direct NVMe Host path is simpler and lower-latency than any fabric. Add a network only when you genuinely need reach or pooling.
- The network cannot meet the workload. A fabric endpoint is only as good as the link behind it; a congested or under-provisioned network will bound your throughput regardless of the IP.
- The transport is over-specified. Reaching for an RDMA fabric when NVMe/TCP would comfortably meet the latency budget adds hardware cost and integration effort for no benefit — and the reverse is true if the budget is tight.
The honest test is the same as for any storage decision: start from the data rate, the latency budget and where the data must physically live, then choose the transport and topology that meet them with the least complexity.
Need an NVMe host or storage IP core for your FPGA? Our engineers ship production-proven cores for AMD and Altera devices.
Frequently asked questions
What is NVMe over Fabrics in simple terms?
It is a way to use the NVMe storage command set across a network instead of only over a local PCIe link. A host can read and write NVMe namespaces that live on a remote machine or node, with latency designed to stay close to that of a directly attached drive.
How is NVMe-oF different from plain NVMe?
Plain NVMe talks to a drive over PCIe inside one machine. NVMe-oF carries the same commands, queues and completions over a network transport, so the storage can be somewhere else on the fabric while the host still sees ordinary NVMe namespaces.
What is the difference between a target and an initiator?
The target owns the storage and exposes its namespaces onto the fabric. The initiator (host) consumes that storage, discovering targets and issuing reads and writes as if the drives were local.
Which transport should I choose — RDMA, TCP or Fibre Channel?
RDMA (RoCE, iWARP, InfiniBand) gives the lowest added latency but needs RDMA-capable hardware. NVMe/TCP runs on ordinary Ethernet with no special NICs and is the simplest to deploy. NVMe/FC suits sites with an existing Fibre Channel SAN. The right choice follows your latency budget and the network you already have.
Why implement NVMe-oF on an FPGA instead of in software?
An FPGA can run the transport and NVMe-oF protocol in hardware along a deterministic, CPU-less path. That removes operating-system jitter and offloads the work from any host processor, which matters for high-rate, latency-sensitive storage and recording.
Where does the LDS NVMe SAN IP fit in a topology?
It sits at the target/initiator boundary of an NVMe-oF fabric, letting an FPGA present its storage onto the network or reach storage elsewhere on it. It extends the reach of the local NVMe Host IP path from a directly attached drive out onto the fabric.
Planning an FPGA storage node that has to join a fabric rather than own its drives? Tell us your transport and latency targets and we will help you confirm the endpoint design is sound.
