NVMe Explained: Queues, Namespaces and the Host Controller

NVMe Explained: Queues, Namespaces and the Host Controller

Industry update — NVM Express ratified the NVMe 2.4 specification set on 31 July 2026 and published it on 4 August 2026 — the Base Specification, the I/O command-set documents and the PCIe, RDMA and TCP transport specifications all released together as one versioned set (NVM Express, 4 August 2026).

In short: NVMe is a queue protocol, not a connector. The host and the SSD share circular submission and completion queues in memory; the host rings a doorbell register to say “commands are waiting”; namespaces divide the drive into addressable units; and a separate Admin queue pair configures everything else. PCIe is only the transport underneath.

Most explanations of NVMe start from the M.2 connector, which is the wrong end. NVMe is a command protocol defined by NVM Express, Inc., and its design fits in one sentence: put the command queues in memory, let each side own one pointer, and get the operating system out of the way. For a hardware engineer that is good news — circular buffers and pointer arithmetic are things you can implement in fabric, with no processor in the path. This guide covers the pieces an FPGA engineer has to reason about, referenced throughout to the published specification. For the implementation side, start from our complete guide to NVMe Host IP cores for FPGA.

The short answer

NVMe pairs a Submission Queue (SQ) with a Completion Queue (CQ), both circular buffers in host memory. The host writes a 64-byte command into the next free SQ slot, then writes the new tail value into a 32-bit SQ Tail Doorbell register on the controller. The controller fetches entries in order, may execute them in any order, posts a 16-byte completion entry and flips a phase-tag bit so the host can see it is new; the host acknowledges by writing the CQ Head Doorbell. One mandatory Admin queue pair configures the controller and creates the I/O queue pairs, and namespaces identify which slice of the drive a command targets. That loop — command, doorbell, completion, doorbell — is the protocol.

NVMe is a protocol, not a connector

The commonest misconception is that NVMe is a physical interface the way SATA is. It is not. The NVM Express Base Specification — revision 2.4 at the time of writing — defines “a protocol for host software to communicate with non-volatile memory subsystems over a variety of memory-based transports and message-based transports.” Connectors and electrical layers belong to the transport specification underneath. Knowing the layers saves time when a detail is not where you expect it:

  • Base Specification — controller model, queue mechanics, Admin Command Set, property (register) map. It answers “how does a command get executed?”
  • I/O Command Set specifications — what commands do. The NVM Command Set is the familiar block one; Zoned Namespace, Key Value, Computational Programs and SLM are separate.
  • Transport specifications — PCIe, RDMA and TCP each have their own. The doorbell registers are defined in the PCIe transport document, not the Base spec.
  • Boot and Management Interface (NVMe-MI) — boot constructs and out-of-band subsystem management.

That layering is why the same command model runs unchanged over a local PCIe link and over a 100 GbE fabric, and why NVMe over Fabrics on FPGA is not a second protocol to learn.

The queue pair: submission, completion, doorbell

Everything interesting happens inside a queue pair. A Submission Queue is, in the specification’s words, “a circular buffer with a fixed slot size that the host uses to submit commands for execution by the controller”; the Completion Queue is its mirror. Both live in host-allocated memory, and the controller holds only pointers into them.

Step 1 — the host builds a command

Each submission queue entry is a command, and commands are 64 bytes: an opcode, a Namespace Identifier naming the target, a command identifier for matching the completion later, and pointers to the data buffer. Those pointers are either PRP entries (Physical Region Page, a page-granular pointer list) or a Scatter Gather List segment. A command carries at most two PRP entries inline; if the buffer needs more, one points at a PRP List instead. This is where the storage protocol meets your DMA engine — most of the work in a hardware implementation.

Step 2 — the host rings the doorbell

Writing the command into memory does nothing on its own. The controller is alerted only when the host writes the SQ Tail Doorbell register with the new tail value, and the difference between old and new tells it how many commands were just submitted — which is why one doorbell write can publish a whole batch. Each doorbell is 32 bits, and the spacing is not fixed: CAP.DSTRD advertises a stride of 2^(2+DSTRD) bytes, so a compliant host computes doorbell addresses rather than assuming 4-byte packing.

Step 3 — the controller executes, out of order

The controller “fetches SQ entries in order from the Submission Queue and may execute those commands in any order.” That sentence is why NVMe scales: the queue is a submission mechanism, not an execution order. If you need ordering between two commands, enforce it yourself — wait for the first completion, or use the fused-operation features the spec defines. Assuming the drive preserves submission order is a classic and expensive design error.

Step 4 — the completion and the phase tag

A completion queue entry is 16 bytes: a two-byte status field, the identifier of the command it completes, and the current SQ head — which is how the host learns SQ slots have been freed without a second round trip. How does it know an entry is new? The controller inverts a single Phase Tag bit each time it wraps the queue, and the host polls that bit. An interrupt may also be raised, but it is an optimisation, not a requirement — which is why a polled hardware host needs no interrupt controller at all. The host then writes the CQ Head Doorbell to release the slots.

Building a storage path that has to run this loop in fabric rather than in software? Our NVMe Host IP cores implement it for AMD, Altera and Microchip families.

See the NVMe Host IP core range

Queues and structures at a glance

Every figure below comes from the NVM Express Base Specification, revision 2.4. Mind the gap between what the protocol permits and what a real controller advertises: the ceilings are architectural, not typical.

NVMe queue and command structures, as defined by the specification
Structure Defined size or limit What it does
Submission Queue (SQ) Circular buffer, fixed slot size Host-owned ring where commands are placed for the controller to fetch.
Completion Queue (CQ) Circular buffer, fixed slot size Ring where the controller posts results back to the host.
Submission queue entry (SQE) 64 bytes One command: opcode, NSID, command ID, and PRP or SGL data pointers.
Completion queue entry (CQE) 16 bytes, incl. 2-byte status One result, plus the SQ head pointer and the phase tag.
I/O queues Up to 65,535 I/O Queues Parallel data-path queue pairs, typically one per host core or per user.
Outstanding commands Up to 65,535 per I/O Queue Architectural depth ceiling; the real limit is CAP.MQES, advertised per controller.
Admin queue pair Exactly one per controller Mandatory management channel; creates and deletes the I/O queues.
Doorbell register 32 bits, stride 2^(2+DSTRD) bytes The only signal the host sends the controller in the fast path.
Namespace “A set of resources that may be directly accessed by a host” The addressable unit a command targets, selected by its NSID.

The 65,535 × 65,535 figure is often rounded to “64K queues, 64K deep”; treat it as headroom. What your design must honour is the CAP.MQES value the attached controller reports — a zero-based field whose minimum legal value means two entries.

Admin queue vs I/O queues

NVMe splits its command space in two, along the control-plane / data-plane line hardware engineers already think in. Every controller implements one Admin Submission Queue and one Admin Completion Queue, without exception. The pair is bootstrapped from three controller properties — AQA (queue sizes), ASQ and ACQ (base addresses) — programmed before the controller is enabled. Only Admin Command Set commands may be submitted there: Identify, Get Log Page, Set Features, Format NVM, Namespace Management and, critically, Create I/O Submission Queue and Create I/O Completion Queue.

The I/O queues are what the Admin queue creates, and they carry the I/O Command Sets. That gives a natural bring-up order: enable the controller, run Identify, create the I/O queue pairs you need, then move data. It is also a natural partition in fabric — the Admin path a low-rate register-driven state machine, the I/O path a wide, latency-sensitive datapath.

NVMe namespaces, briefly

The Base Specification defines a namespace tersely: “a set of resources that may be directly accessed by a host (e.g., formatted non-volatile storage).” For block storage, read that as a range of logical blocks with its own capacity and format, presented as an independent unit.

Every command carries a Namespace Identifier (NSID) naming its target, and the specification is strict about mistakes: an inactive NSID aborts the command with Invalid Field in Command, an invalid NSID with Invalid Namespace or Format — two error codes for two different errors, useful when debugging a host that has just come up.

Why they exist: one physical SSD can present several independent storage units, with different block sizes or protection settings, to one or several controllers. That underpins multi-tenant drives, shared namespaces, and partitioning a drive between independent capture channels. The full picture — management, sharing, multi-drive layouts — is in our guide to NVMe namespaces and multi-drive architectures on FPGA.

NVMe over PCIe, and the stack it replaced

Over PCIe the controller exposes its properties — capabilities, configuration, status, the Admin queue registers and the doorbell array — through a memory-mapped BAR. The host reads CAP, programs CC and the Admin queue properties, sets the enable bit and waits for ready in CSTS. Data never travels through those registers: it moves by DMA, the controller reading and writing the buffers the PRP entries or SGLs point at. Link speeds and lane counts come from the PCI-SIG PCI Express specifications, not from NVMe.

Contrast the interface NVMe displaced. The legacy AHCI model was built for a rotating disk: one command queue, shallow depth, a register-and-interrupt handshake per command. NVMe’s answer is not a faster version of the same thing but a different topology — many independent queue pairs so parallel requesters never contend, and a doorbell write that publishes a batch at once. We take that comparison apart in NVMe vs AHCI, and the interface-selection question in SATA vs NVMe for FPGA storage.

What changes when the host is an FPGA

Read the protocol again with an implementer’s eye and something becomes obvious: nothing in the queue model requires software. A circular buffer, two pointers, a 32-bit register write and a polled status bit all map directly onto fabric. That is what a hardware NVMe host does — the whole loop in logic, so an FPGA drives a commercial SSD at line rate with no processor, driver or operating system in the path.

The trade-offs shift accordingly. In our own NVMe Host IP cores the queue counts are deliberately not the specification’s ceilings: the cores implement up to 16 independent I/O queues and 8 namespaces. A general-purpose OS wants a queue per CPU core to avoid lock contention; an FPGA recorder has a handful of producers and no locks to contend over. Sizing to the real workload is what keeps the logic footprint sane.

The same reasoning shows elsewhere: an I/O queue buffer selectable from 32 KB to 1024 KB, a sector size of 512 B or 4096 B to match the drive, and two host-side interfaces — a CPU-style register interface for sequential access, a FIFO interface for I/O-intensive streaming. On AMD devices the control registers are AXI4-Lite with an AXI4 data path; on Agilex 7 R-Tile the binding is Avalon; on PolarFire recorders, APB. Different buses, identical protocol underneath.

Where this explanation stops

This is the protocol core, deliberately — not the whole specification. A few boundaries worth naming:

  • Namespaces get one section, not the full treatment. Management, sharing, multi-path and multi-drive layouts each have real depth — see the dedicated namespaces guide.
  • No performance numbers. Throughput depends on the drive, the PCIe generation and lane count, the queue depth you sustain and your buffering — not on the protocol. Measure on your own hardware.
  • Resource cost is device-specific. LUT, FF and BRAM utilisation depends on the core, the FPGA family and the configuration. Take those from the datasheet and your own implementation run.
  • Real drives vary. SSDs differ in what they advertise in CAP and how many queues they will create. Read the identify data at run time; do not hard-code assumptions.

In summary

NVMe is easier to reason about than its reputation suggests, because the mechanism is small. The host writes 64-byte commands into a shared ring and rings a doorbell; the controller fetches them in order, executes them in any order, posts 16-byte completions and flips a phase bit; the host rings a second doorbell to free the slots. A mandatory Admin queue pair creates the I/O queue pairs, and every command names its namespace. That model is what lets tens of thousands of commands be outstanding at once — and what lets the whole thing run in FPGA fabric with no software in the data path.

Frequently asked questions

What is the difference between a submission queue and a completion queue?

A Submission Queue is a circular buffer in host memory where the host places 64-byte commands for the controller to fetch; a Completion Queue is the matching ring where the controller posts 16-byte results. They are separate structures with separate doorbells, and need not be paired one-to-one — several submission queues may share one completion queue. The host owns the SQ tail and the CQ head; the controller owns the SQ head and the CQ tail. Each side advances only its own pointer, which is what makes the exchange lock-free.

How many queues does NVMe actually support?

The NVM Express Base Specification allows up to 65,535 I/O Queues, each with up to 65,535 outstanding commands, plus one mandatory Admin queue pair per controller. Those are architectural ceilings, not typical values. What matters in a real design is the CAP.MQES field, which each controller advertises as the maximum queue size it supports. Embedded hosts implement far fewer by choice: our NVMe Host IP cores implement up to 16 independent I/O queues, which covers multi-user and multi-channel FPGA workloads without spending logic on queues nothing will use.

What is an NVMe doorbell register?

A doorbell is a 32-bit register in the controller’s memory-mapped space that the host writes to publish a new queue pointer. Writing the SQ Tail Doorbell tells the controller commands are waiting, and the difference from the previous value tells it how many; writing the CQ Head Doorbell tells the controller which completion entries the host has consumed, freeing those slots. Doorbell registers are defined in the NVMe over PCIe Transport Specification, not the Base Specification, and are not necessarily packed 4 bytes apart — CAP.DSTRD gives a stride of 2^(2+DSTRD) bytes, so addresses must be computed.

What is an NVMe namespace?

The specification defines a namespace as “a set of resources that may be directly accessed by a host”, which for block storage means a range of logical blocks with its own capacity and format, presented as an independent unit. Every command carries a Namespace Identifier naming its target. One SSD can expose several namespaces — different block sizes, different protection settings, or separate areas for separate users — and a namespace can be shared by more than one controller. In an FPGA recorder they are a clean way to keep independent capture channels apart on a single drive.

Do you need a CPU and a driver to use NVMe?

No. NVMe is a memory-and-register protocol: circular buffers, pointer arithmetic, 32-bit register writes and a polled status bit. All of that maps onto FPGA logic, which is what a hardware NVMe host core does — it runs the full command and completion loop in fabric, so a commercial SSD can be driven at line rate with no processor, driver or operating system in the data path. That matters most in deterministic embedded systems, where an OS storage stack is exactly the source of jitter you were trying to remove.

Is NVMe tied to PCIe?

No. PCIe is one transport binding among several. The Base Specification defines the protocol over “memory-based transports and message-based transports”, and separate documents bind it to PCIe, RDMA and TCP. The queue-pair, doorbell and namespace model is identical in every case — what changes is how a command travels: a memory-mapped doorbell write over PCIe, or a capsule sent over a network for NVMe over Fabrics. A team that has implemented an NVMe host over PCIe already understands most of a fabrics implementation.

LDS
Logic Design Solutions
FPGA design house and soft-IP core vendor — NVMe, SATA, ARINC 429, 10G Ethernet MAC and exFAT IP for AMD, Altera and Microchip FPGAs since 1994. IP delivered in VHDL/Verilog source; DO-254 documentation available on request.

How we write these guides: each is drafted by the engineers who build and support our storage cores. Protocol behaviour is quoted from the published NVM Express specifications and linked; product capabilities only where published on our own datasheets. Throughput, latency and resource utilisation are always deferred to the target core’s datasheet and your own implementation results — never to a made-up figure.

Need an NVMe host running in fabric — recorder, SAN node or embedded storage controller? Tell us your FPGA family, PCIe generation and sustained rate, and we will point you to the right core or take on the design.

Browse the IP core catalogue

  1. NVM Express, Inc. — NVMe Specifications: the specification set, including the 2.4 release published 4 August 2026.
  2. NVM Express Base Specification, Revision 2.4 (ratified 31 July 2026, PDF): source for the queue model, the 64-byte SQE and 16-byte CQE, the 65,535 limits, CAP.MQES, CAP.DSTRD, the Admin queue properties and the namespace and NSID definitions.
  3. NVM Express, Inc. — NVMe over PCIe Transport Specification: where the doorbell registers and the PCIe property mapping are defined.
  4. NVM Express, Inc. — NVM Command Set Specification: the block-storage I/O command set carried on the I/O queues.
  5. PCI-SIG — PCI Express Specifications: the transport underneath NVMe over PCIe, which sets link speeds and lane counts.
  6. Wikipedia — NVM Express: revision history of the specification set.