News : In May 2025, Hexion Storage Networks announced that it had built Africa’s largest All-NVMe SAN backbone with H3C. For readers examining NVMe SAN architecture, the announcement is a concrete case to consider: a named All-NVMe SAN build, not a claim that one architecture fits every storage environment. (ITWeb, 14 May 2025).
An NVMe SAN architecture is easier to review when the design makes three boundaries explicit: what the FPGA exposes, which transport carries it, and where each function belongs. We recommend using those boundaries to organize decisions about queues, paths, node ownership, and validation.
The short answer
Define the FPGA’s storage-network interface first, then select a fabric transport that matches the intended endpoints.
RDMA, TCP/IP, and Fibre Channel are distinct options. Record the selected transport beside the namespace it carries.
We recommend assigning each function to FPGA logic, host software, or the storage target before refining implementation details, and treating queue scope, path selection, and node ownership as separate decisions. Keep them separate. Then validate the resulting design with a consistent workload and comparable measurements.

What does an NVMe SAN data path look like on an FPGA?
An FPGA NVMe SAN data path is best shown as an accelerator exposing an NVMe namespace, with the selected transport labeled separately, as SNIA’s description of an FPGA accelerator using NVMe over Fabrics explains.
Keep the view at the interface. Identify the accelerator, draw the namespace boundary explicitly, and label the transport selected for the design. Use that boundary to separate what the FPGA exposes from transport configuration, so the sketch stays focused on the interface rather than implying details the design has not specified. Leave those configuration details out.
Transport labels need to be specific. SNIA names RDMA, TCP/IP, and Fibre Channel as transports for sharing the namespace; treat them as design choices, not interchangeable labels. Let endpoint and integration requirements guide the choice. Then use the same transport name beside the namespace and in the interface notes, keeping the drawing consistent with its accompanying description.
Keep performance language equally precise. Electronic Design describes NVMe as a streamlined means of delivering low-latency operation and says NVMe-over-Fabrics could help administrators use available memory resources more efficiently. We frame latency and memory use as design goals, not automatic outcomes. The cited benefits are possibilities, not results established for every implementation, so evaluate them against the chosen design.
A product reference can sit beside the generic interface sketch without defining how the design divides its functions. Logic Design Solutions identifies its NOE SAN IP as a product line built on its NVMe and Ethernet cores. Use that reference to connect the architecture discussion to the stated IP offering; keep the namespace and transport labels specific to the design being discussed.
Which NVMe functions belong in FPGA logic, host software, and storage targets?
Assign ownership explicitly: define which functions belong in FPGA logic and host software, and keep the storage target’s block-device presentation role distinct.
In the documented SPDK NVMe-oF target implementation, a user-space application presents block devices to hosts over a fabric such as Ethernet, InfiniBand, or Fibre Channel; treat that implementation as an example of the target’s role, not as a required allocation for every architecture.
| Design area | Responsibility to define |
|---|---|
| FPGA logic | Name the functions assigned to FPGA IP and specify the interface at its boundary. |
| Host software | Identify software-owned functions and document their handoff to or from FPGA logic. |
| Storage target | Define its role in block-device presentation, distinct from FPGA and host-software responsibilities. |
We recommend starting with ownership.
Record the interface between each owner, keep block-device presentation explicit at the target boundary, and avoid leaving a function with unclear or overlapping ownership. When a responsibility is split between FPGA logic and host software, describe that split rather than assigning the whole function to both.
For each function, record its owner, inputs and outputs, and the boundary it crosses. Keep the storage-target role distinct in the design description, so its presentation of block devices is not confused with choices about FPGA logic or host software. An ownership map gives the team a practical way to discuss implementation without assuming that one allocation fits every design.
How should the bus fabric connect clustered FPGA nodes?
Size the fabric against intended I/O demand. A NSDI 2026 paper on traffic control with NVMe-oF notes that new NVMe form factors are making commodity storage appliances denser, which calls for SAN fabrics to expand their bandwidth capacity.
For the capacity decision, use the workload assumption as the basis, not node density alone. Our recommendation is to treat density as a review trigger, not a sizing value. Keep the node map, workload assumption, and capacity target together in the design record, so the rationale stays attached to the design case.
Make the architecture diagram tell the same story. Identify which nodes belong to the design case and which interfaces are assigned to the path. Place the capacity target beside the I/O-demand assumption, keeping the workload assumption and capacity target together. The node map, demand assumption, and target should be easy to read as parts of one design decision, rather than as separate inputs with no stated relationship.
When density changes, revisit the workload assumption and capacity target together, then decide whether the design case still supports the same rationale or needs an explicit update in the design record. If the demand case stays the same, retain the existing rationale instead of deriving a new target from density alone. Record any change to that rationale explicitly.
How should a cluster coordinate NVMe queues, paths, and node ownership?
Coordinate a cluster by recording queue scope, path selection, and node ownership as separate decisions, with each entry naming the relationship or assignment it describes.
Start with the queue entry’s scope. SNIA’s explanation of NVMe multi-queue scope describes host-controller relationships for both PCIe-attached and fabric-connected controllers. Name which relationship a queue entry covers, then identify the connection type. Keep distinct relationships in separate entries instead of folding them into one broad label. Keep scope explicit.
Path selection belongs in its own entry. The Linux multipath documentation describes policies that follow ANA and prefer an optimized path when one is available. State whether Linux multipath is part of the design, and keep the availability condition beside the preference. A preference is not an unconditional promise. Keep the condition attached.
Ownership needs a different set of fields: name the resource, the node assigned to it, the assignment’s scope, and the design condition that would change that assignment. If the design uses labels such as “active” or “primary,” define what each means in that design. Otherwise, leave those labels out; they do not specify the ownership rule on their own.
Our recommendation is to keep these decisions in three separate entries: the host-controller relationship for each queue, the path policy and its availability condition, and the resource’s node assignment and scope. Record the condition that changes the assignment instead of leaving it implicit. A reviewer can then read each entry for one decision, rather than infer scope, path behavior, or ownership from a label intended to describe something else.
Which fabric and datapath criteria should drive the architecture choice?
Our recommendation is to choose a fabric from the intended endpoints’ transport requirements, not from the NVMe label alone; the NVM Express specifications cover communication over multiple transports, including PCIe, RDMA, and TCP.
Treat NVMe as protocol context, not as a complete fabric decision. Start by recording each intended endpoint and the transport the design expects it to support. Endpoint support should drive the fabric choice.
When the transport is still open, leave it open in the architecture description and distinguish confirmed design inputs from assumptions, so the datapath notes do not quietly turn an unresolved decision into a fixed one. Keep the decision visible.
- Record the intended endpoints and their transport requirements.
- Compare candidate fabrics against those requirements; do not infer the transport from a shared NVMe label.
- Name the selected transport at the datapath boundary, then use the same label in the block diagram and interface notes.
If the endpoints do not share a supported transport choice, make that constraint explicit before settling the fabric. When PCIe matches the intended endpoints, carry PCIe into the design description; if RDMA or TCP is the match, use that label instead. Keep the fabric decision tied to the endpoints, rather than presenting one option as universal.
How should engineers validate latency, throughput, and scaling?
We recommend validating latency, throughput, and scaling with a consistent workload matrix, holding test conditions steady across configurations and changing one design variable at a time, thereby keeping comparisons interpretable.
Keep the basis fixed. Record the request pattern and configuration under test, then use the same definitions for latency and throughput; present latency distribution alongside throughput, so the design review does not depend on a single summary value, and include both per-node and aggregate throughput when comparing scaling points.
A benchmark-based evaluation gives a concrete example: an NVMe-oF architecture study, published in Electronics in June 2023, describing an implementation of SPANoF in the Linux kernel evaluated with FIO benchmarks. Use it as an example of benchmark-based evaluation, not as a performance result for a different design.
For each run, record the settings and duration, and present variation rather than hiding it in a single summary figure. Keep the workload matrix consistent across configurations; change the selected configuration in controlled steps. The aim is to give engineers the workload and measurement definitions alongside the results.
For a particular IP configuration, identify it in the results and label each result with its test conditions. The useful output is a clear record of latency distribution, per-node and aggregate throughput, and results at each scaling point.
When is a SATA-based storage path a better fit than an NVMe SAN?
A SATA-based storage path is a better fit when the required rate matches one of its listed speeds, 6.0 Gb/s, 3.0 Gb/s, or 1.5 Gb/s, according to SATA-IO’s listed SATA transfer speeds.
We recommend using the required rate as an interface-level screen, then comparing it with SATA-IO’s listed transfer speeds, because a match keeps SATA in consideration without establishing that it is right for the complete system.
Assess NCQ separately. SATA-IO identifies Native Command Queuing (NCQ) as a supported feature, but feature support alone does not establish a queue depth or a performance result. SATA merits consideration when NCQ aligns with the interface requirements.
Frequently asked questions
What is the NVMe architecture and how does it work?
For an FPGA-based NVMe SAN, we recommend showing the accelerator, the standard NVMe namespace it exposes, and the selected transport at the fabric boundary. SNIA describes sharing that namespace over RDMA, TCP/IP, or Fibre Channel. Keep the diagram focused on those boundaries rather than implying transport-configuration details.
How does NVMe work?
In the architecture described here, the FPGA accelerator presents a standard NVMe namespace and a selected fabric transport carries it between endpoints. The NVM Express specifications cover communication over transports including PCIe, RDMA, and TCP. We recommend naming the transport in the design instead of relying on the NVMe label alone.
Is NVMe over TCP?
Yes. SNIA names TCP/IP as an option for sharing an NVMe namespace over a fabric. We recommend documenting TCP/IP as the selected transport only when it matches the endpoint requirements; the cited description does not specify how to configure it.
In summary
We recommend making the namespace boundary, fabric choice, and function ownership explicit in the architecture description. Keep queue scope, path policy, and node ownership distinct, and tie the capacity rationale to the workload assumption. For validation, define the workload and measurement terms before comparing configurations, then keep the results with the conditions that produced them.
How we wrote this article
The sources we consulted are listed at the end of the article, consulted on 2 October 2026. Where we cite none, it is our opinion, and we write it as such.
This article gives buying criteria and our point of view. It does not replace a professional’s advice on a specific situation.
It is published by Logic Design Solutions, which therefore has an interest in the subject.
Sources
- Determining the Best Option for NVMe-over-Fabrics | Electronic Design, electronicdesign.com
- FPGA Accelerator Disaggregation Using NVMe-over-Fabrics | SNIA | Experts on Data, snia.org
- Co-Designing Traffic Control with NVMe-oF for Disaggregated Storage: A Comparative Study of Switched and Switchless SAN , usenix.org
- Questions Aplenty on NVMe over Fabrics | SNIA | Experts on Data, snia.org
- Linux NVMe multipath: The Linux Kernel documentation, cdn.kernel.org
- Specifications – NVM Express, nvmexpress.org
- SPANoF: A Scalable and Performant Architecture for NVMeoF-Based Storage Disaggregation with Limited Network Resources, mdpi.com
- The SATA Ecosystem | SATA-IO, sata-io.org
- What Is NVMe SAN IP? Building Storage Networks in FPGA Fabric – Logic Design Solutions | Logic Design Solutions, logic-design-solutions.com
- SPDK: NVMe over Fabrics Target, spdk.io
