PCIe on FPGA Explained: Endpoints, Root Complex and NVMe

PCIe on FPGA Explained: Endpoints, Root Complex and NVMe

Industry news — On 25 August 2026 AMD detailed its Versal Premium Gen2 adaptive SoC at Hot Chips 2026, describing PCIe Gen1 through Gen6 SerDes plus Gen7 signalling taken directly into the programmable logic at up to 64 GT/s, and CXL 4.0 brought straight to the fabric at up to 128 GT/s — which is a reminder that on an FPGA the PCIe generation you can reach is a property of the silicon you picked, not of the IP you licensed (ServeTheHome, 25 August 2026).

In short: on an FPGA, PCIe is a hard block in the silicon and the soft IP above it only chooses the role — and hosting an NVMe SSD means the FPGA has to be the Root Complex, not the Endpoint that almost every vendor example design assumes.

That announcement makes a point most PCIe discussions skip: the generation is decided in the silicon, long before anyone opens a datasheet for an IP core. But there is a second decision, made just as early and noticed just as late, and it is the one that actually breaks storage projects. PCI Express is an asymmetric protocol. One device in the tree enumerates, allocates and configures every other device — and if your FPGA has to drive an NVMe SSD, that device has to be your FPGA. Nearly every FPGA PCIe example design, reference project and application note assumes the opposite. This article explains the PCIe model an FPGA engineer actually needs: the topology, configuration space and BARs, the three protocol layers, the PCI-SIG generation ladder, what the hard block gives you on AMD, Altera and Microchip parts, and what the Root Complex role costs when there is no processor to do it for you.

The short answer

PCI Express is a point-to-point, packet-switched tree with exactly one device at
the top: the Root Complex. Everything below it — switches, bridges and
Endpoints — is discovered, addressed and configured by that root. An FPGA sitting
in a server slot is an Endpoint, and that is the shape of almost every vendor example design. An
FPGA that has to host an NVMe SSD is the opposite: it must be the Root Complex, drive the
drive out of reset, train the link, walk configuration space, size and program the
Base Address Registers, enable bus mastering and set up MSI-X — all the work a BIOS
and an operating-system PCIe subsystem normally do in software. On an FPGA with no processor in the
path, that work has to exist as logic, which is precisely why an NVMe host IP core is a product
rather than a weekend integration job.

The PCIe topology model: one root, many Endpoints

A PCI Express fabric is a tree of point-to-point links, not a shared bus, despite inheriting
PCI’s addressing model and terminology. Each link connects exactly two ports, and each link is made
of one or more lanes: a lane is a dual-simplex pair of differential signals, one pair per
direction. A x4 link therefore carries four lanes in each direction, and the two ends negotiate the
width they will actually use at training time.

Three device roles matter. The Root Complex is the top of the tree and the only
entity that can originate configuration transactions; it owns the system memory map and, in a
conventional computer, sits inside the CPU package. A switch has one upstream port
facing the root and several downstream ports facing further devices; in configuration terms a switch
looks like a set of PCI-to-PCI bridges. An Endpoint is a leaf: a network card, a
GPU, an NVMe SSD, or an FPGA accelerator card. Endpoints never enumerate anything. They answer
configuration reads, expose their capabilities, and wait to be told where in the memory map they
live.

This asymmetry is the whole story of this article. The names “host” and
“device” that engineers borrow from SATA or USB map onto PCIe as Root Complex and
Endpoint, and the mapping is not a convention you can quietly ignore — it changes which logic
you have to build. An SSD is always an Endpoint. It will never enumerate your FPGA. If the SSD is
going to work, something upstream of it has to behave like a root.

Configuration space, enumeration and BARs

Every PCIe function exposes a configuration space, and enumeration is the process by which the
Root Complex walks the tree, reads that space and fills in the parts that describe where the
function lives. This is the mechanism that turns a collection of silicon into an addressable
system.

Bus, device, function

PCIe keeps PCI’s bus-device-function addressing: an 8-bit bus number, a 5-bit device
number and a 3-bit function number identify one function anywhere in the tree. The root starts at
bus 0, reads the vendor and device ID of each candidate, and when it finds a bridge it assigns that
bridge a secondary bus number, recurses into it, and records the highest bus number found beneath as
the subordinate bus number. That recursive walk is not conceptually hard — but it is stateful,
it involves arithmetic on a resource map, and it is normally done by firmware.

The header and the capability list

The first 256 bytes are the PCI-compatible header; PCIe extends the space to 4096 bytes per
function for extended capabilities such as Advanced Error Reporting. A Type 0 header describes an
Endpoint; a Type 1 header describes a bridge and carries the primary, secondary and subordinate bus
number fields. The class code identifies what the function is, which is how a host recognises an
NVMe controller without knowing the vendor. Beyond the header sits a linked list of capability
structures — power management, PCI Express capability (link speed and width, both
“capable” and “current”), MSI and MSI-X.

BARs and the memory map

A Base Address Register declares that a function wants a window in the host
address space. The size is discovered rather than declared: the root writes all ones to the BAR,
reads it back, and the number of low-order bits that stayed zero tells it how large the window is
and whether it is 32-bit or 64-bit, memory or I/O, prefetchable or not. The root then picks a base
address that does not collide with anything else and writes it back. Only after that, and after the
Bus Master Enable and Memory Space Enable bits are set in the command register, can the function be
reached or initiate traffic of its own.

The Root Complex inversion: what it costs on an FPGA

An FPGA that hosts an NVMe SSD is the Root Complex, and this single inversion is where most
storage projects lose a month. The default mental model — reinforced by evaluation boards,
driver examples and DMA reference designs — is the FPGA as an accelerator card plugged into a
server: the FPGA is the Endpoint, the server enumerates it, and a Linux driver pokes its BARs. Flip
the system around so the SSD hangs off the FPGA, and every one of those assumptions inverts.

Concretely, the FPGA now has to do the following before a single NVMe command exists:

  • Hold the drive in reset and release PERST# with correct timing relative to
    power and reference clock stability, then wait for the link to train.
  • Own a memory map. There is no operating system to arbitrate address space, so the design must
    decide statically or in logic where the SSD’s BAR windows land.
  • Issue configuration reads and writes — transactions only a root port can generate —
    to read the class code, size the BARs, program them, and set the command register bits.
  • Configure interrupts. An NVMe controller expects MSI-X: the root has to read
    the MSI-X capability, locate the vector table in BAR space, program the message address and data for
    each vector, and enable the function.
  • Then, and only then, start the NVMe layer: write the admin queue base addresses, set the
    controller configuration register, wait for ready, and identify the controller and its
    namespaces.

In a PC, everything above the last bullet is BIOS plus the kernel’s PCIe subsystem — tens
of thousands of lines of software written once and reused everywhere. On a CPU-less FPGA it has to
exist as a state machine, or as firmware on a soft processor you added only for this. That is the
honest answer to “why does an NVMe host IP core exist at all”: it is not the NVMe
command set that is expensive, it is the root-side plumbing underneath it. Logic Design Solutions
publishes automatic PCIe root-port or endpoint configuration and automatic NVMe configuration as
features of its NVMe Host IP for exactly this reason. Our
complete guide to NVMe Host IP for
FPGA
covers what such a core contains, and
connecting an NVMe Host IP to the
fabric over AXI
covers the interface it presents once the root-side work is done.

The three protocol layers, and which one bites you

PCIe splits into three layers, and each fails in a recognisably different way. Knowing which
layer a symptom belongs to is most of the debugging.

Transaction layer

The transaction layer carries Transaction Layer Packets (TLPs). The types you will
meet are memory read, memory write, completion (with or without data), configuration read and write,
and messages. A TLP header is three or four double-words depending on whether the address is 32-bit
or 64-bit, and it carries a requester ID and a tag so that a split completion can be matched back to
its request. Memory writes are posted — fire and forget. Memory reads are non-posted: the
requester sends a read and waits for one or more completions, which is why read latency, not write
latency, dominates a PCIe design. Two negotiated parameters live here and quietly decide your
efficiency: Max Payload Size and Max Read Request Size. A link
where one end is stuck at a 128-byte payload will move data, and it will move it badly.

Data link layer

The data link layer makes the link reliable. Every TLP gets a sequence number and an LCRC, is
held in a replay buffer, and is acknowledged or negatively acknowledged by the far end; a NAK or a
timeout triggers replay. It also runs credit-based flow control, separately for posted, non-posted
and completion traffic, and separately for headers and data. Credits are advertised at link
initialisation and returned as buffers free up. A receiver that advertises few completion credits
throttles the whole read pipeline, which is the classic reason a link that has trained at full width
and full speed still delivers disappointing throughput.

Physical layer

The physical layer owns the SerDes, the encoding and the LTSSM — the Link
Training and Status State Machine that walks through Detect, Polling, Configuration and on to L0,
negotiating lane width, lane order and speed along the way. From Gen3 onward it also runs link
equalisation, a negotiated exchange in which each end asks the other to adjust its transmitter
until the receiver’s eye is acceptable. Encoding changes with generation, which the next section
sets out. When a link comes up at the wrong width or the wrong speed, the LTSSM is where the
evidence is.

Generations and per-lane rates

PCI-SIG defines each PCI Express generation by a per-lane transfer rate and an encoding, and
those two numbers are the only bandwidth facts that are universally true. Everything downstream of
them — MBytes per second in your design — depends on payload size, credits, read depth
and the drive itself.

PCI Express generations as defined by PCI-SIG
Generation Per-lane transfer rate Line encoding Note
PCIe 1.x (Gen1) 2.5 GT/s 8b/10b 20 % encoding overhead
PCIe 2.x (Gen2) 5.0 GT/s 8b/10b Rate doubled, encoding unchanged
PCIe 3.x (Gen3) 8.0 GT/s 128b/130b Encoding overhead drops to about 1.5 %; transmitter equalisation introduced
PCIe 4.0 (Gen4) 16 GT/s 128b/130b Tighter channel loss budget; retimers become common
PCIe 5.0 (Gen5) 32 GT/s 128b/130b Channel budget tighter again
PCIe 6.0 (Gen6) 64 GT/s PAM4 signalling, FLIT mode Four-level signalling with forward error correction; fixed-size flow control units replace the variable framing of earlier generations

Two consequences matter for an FPGA storage design. First, doubling the rate does not double
usable throughput, because the encoding, the flow-control credits and the SSD’s own behaviour all
sit in the way. Second — and this is a rule we apply to ourselves — a MBytes-per-second
figure for a given core on a given generation is a measurement, not an arithmetic
derivation. The only sustained throughput figure Logic Design Solutions has published is roughly
500 MBytes per second over a single 6 Gbit SATA link, announced in 2019. We do not publish a
Gen4 or Gen5 number for an NVMe Host core because we have not published a measurement, and a
calculated one would be a fiction dressed as a specification. If you need a figure for your board,
it is measured on your board. The trade-offs between the two current high-end options are laid out
in Gen4 versus Gen5 for NVMe host IP on
FPGA
.

On an FPGA, PCIe is a hard block

On every current FPGA family, the PCIe controller is a hard block: dedicated silicon next to the
transceivers, implementing the physical and data link layers and much of the transaction layer, with
the soft IP sitting above it in fabric. This is not a detail of implementation, it is the constraint
that decides your project. You do not choose a PCIe generation the way you choose a compiler flag;
you choose a device and a speed grade, and the generation follows. Soft IP can change the role the
block is configured in, the interface it presents to your logic, and the protocol layered on top
— it cannot make a Gen2 transceiver run at 32 GT/s. The distinction between what is hardened
and what you license is covered in
soft IP versus hard IP on FPGA.

Hard PCIe blocks by family — check the specific device and speed grade in the vendor documentation
Vendor and family Hard PCIe block, highest generation offered Fabric interface usually exposed Vendor documentation
AMD Versal Premium Up to Gen5; the Versal Premium Gen2 generation announced at Hot Chips 2026 adds Gen6 SerDes AXI4 / AXI4-Lite amd.com — Versal Premium
AMD Versal Prime and AI Core Up to Gen4 AXI4 / AXI4-Lite amd.com — Versal
AMD UltraScale+ (Zynq, Kintex, Virtex, Artix) Gen3 across the family; Gen4 on selected devices such as Artix UltraScale+ AXI4 / AXI4-Lite amd.com — FPGAs
Altera Agilex 7 Up to Gen5 with the R-Tile; Gen4 with the P-Tile Avalon (streaming or memory-mapped) altera.com — Agilex
Altera Stratix 10 Gen3 with the H-Tile; Gen4 on the P-Tile variants Avalon altera.com — Stratix
Microchip PolarFire and PolarFire SoC Gen2, up to x4, configurable as root port or endpoint AXI4, with APB or AXI4-Lite for control microchip.com — PolarFire

The published device list of a storage IP vendor is, in effect, a map of that table. Logic Design
Solutions documents its NVMe Host IP at Gen2 with a 128-bit datapath at 125 MHz on Kintex 7, Gen3
with a 256-bit datapath at 125 MHz on Kintex UltraScale+, Gen2 at 150 MHz and Gen3 at 250 MHz on
Zynq UltraScale+, Gen4 at 250 MHz with a 256-bit datapath on Artix UltraScale+, and Gen5 on
Agilex 7 with the R-Tile. The
bus follows the vendor too: AXI on AMD parts, Avalon on Agilex 7 R-Tile, APB for the control
interface on the PolarFire SoC
recorders. None of that is a preference. It is the hard block’s interface showing through. If you
are still choosing silicon, how
to choose an FPGA for storage IP across AMD, Altera and Microchip
works through the same
decision from the device side.

How NVMe rides on PCIe — the one-paragraph version

NVMe is a register-level interface mapped into PCIe memory space, and that is genuinely all the
transport does for it. The controller’s registers live in BAR0: the capability and version
registers, the controller configuration and status registers, and an array of
doorbell registers, one pair per queue. The queues themselves — submission
and completion — are circular buffers in host memory, which on an FPGA means memory
the FPGA owns and has published an address for. The host writes a 64-byte command into a submission
queue entry, then writes the new tail index to that queue’s doorbell in BAR space; the controller
issues PCIe memory reads to fetch the command, memory reads or writes to move the data, a memory
write to post a completion entry, and an MSI-X message — itself just a posted memory write to
an address the root programmed — to signal it. Nothing in that flow is PCIe-specific beyond
the fact that every step is a TLP. The queue model, namespaces and the controller state machine are
their own subject, covered in
NVMe explained: queues, namespaces
and the host controller
.

Practical failure modes at PCIe bring-up

Most PCIe bring-up failures are physical or clocking problems wearing a protocol costume. The
link status registers and the LTSSM state are almost always where the evidence is, and reading them
first saves days. These are the ones that recur:

  • The link trains at x1 instead of x4. Usually a lane that never reached receiver
    detect: an unconnected or damaged pair, an AC coupling capacitor missing on one lane, or a
    transceiver channel not enabled in the design. Lane reversal and polarity inversion are handled by
    the specification and are rarely the cause; a lane that is simply not there is.
  • The link drops to Gen1 or renegotiates repeatedly. At Gen3 and above this
    points at equalisation: a channel with more insertion loss than the presets can compensate, a
    missing or misconfigured retimer, or a connector and via stack that was never simulated. A link that
    comes up at full speed and then falls back under load is a signal-integrity margin problem, not a
    protocol problem.
  • Reference clock architecture mismatch. Both ends must agree on how the 100 MHz
    reference is distributed. Common clocking and separate reference clock with no spread spectrum
    behave one way; separate reference clock with independent spread spectrum (SRIS) requires more
    frequent SKP ordered set insertion so the elastic buffers can absorb the frequency offset. Configure
    one end for SRIS and the other for SRNS and the link will train and then produce errors under
    sustained traffic, which is a miserable bug to find late.
  • PERST# and CLKREQ# on the CEM connector. On a card-edge or M.2 socket the FPGA
    is now responsible for the sideband signals a motherboard would drive: asserting and releasing
    PERST# in the right order relative to power rails and reference clock, and handling CLKREQ# if the
    drive uses it. Getting the release order wrong produces a drive that enumerates intermittently or
    only after a warm reset.
  • Hot reset and link-down behaviour. A root port has to decide what happens when
    the link goes down mid-transfer — outstanding reads never complete, queues are stale, and the
    NVMe controller must be re-initialised, not merely re-armed. Designs that never tested a surprise
    removal discover this in the field.

Two of these are timing problems in the fabric rather than on the wire, and the general approach
to those is in
timing closure for high-speed
storage interfaces
. The rest are board problems, and the earlier a PCIe channel is simulated,
the cheaper they are.

Tell us the protocol, the target device and the role your FPGA has to play — root port or endpoint, which generation, how many lanes — and we will tell you what the hard block gives you and what has to be built above it.

Talk to an FPGA engineer

Where this stops applying

This is a model of PCIe for an FPGA engineer building a storage path. Several things sit outside
it:

  • It is not a substitute for the base specification. Ordering rules, error
    handling, ASPM and power state transitions, and the exact field layouts are normative in the
    PCI-SIG documents and nowhere else. Anything you implement is checked against those, not against an
    article.
  • Generation numbers here are ceilings, not guarantees. The highest generation a
    family offers is available on some devices, in some packages, at some speed grades. Confirm the part
    number in the vendor documentation before it reaches a schematic.
  • No throughput figures are given for our cores. Only per-lane transfer rates
    defined by PCI-SIG appear above. A MBytes-per-second number for an NVMe Host IP core on your device
    and your drive comes from measurement on your board, and we will not quote one before it exists.
  • CXL is a different problem. It runs on the same electrical layer and appears in
    the same product announcements, but coherency, memory pooling and the CXL protocol stack are not
    covered here at all.
  • Switched and multi-drive topologies add work. Putting a PCIe switch between the
    FPGA and several SSDs means enumerating bridges, assigning bus number ranges and partitioning the
    memory map — more root-side logic, not a configuration option.

In summary

PCI Express is asymmetric by design, and on an FPGA that asymmetry is the decision
that shapes the project. If the board is an accelerator in a server, the FPGA is an Endpoint and the
vendor’s example design is close to what you need. If the board has to drive an SSD, the FPGA is the
Root Complex, and it inherits enumeration, BAR allocation, MSI-X configuration and link management
— work that is invisible in a PC because BIOS and the kernel did it before anything you wrote
ran. That is the gap an NVMe host IP core fills.

The rest is arithmetic you cannot change: the hard block in your chosen device sets the
generation, PCI-SIG sets the per-lane rate for that generation, and the encoding, credits and the
drive decide what fraction of it you keep. Choose the silicon with the role and the generation
already in mind, capture the LTSSM state early, and treat every throughput number as something to be
measured rather than derived.

Frequently asked questions

What is the difference between a PCIe Root Complex and an Endpoint?

The Root Complex is the top of the PCIe tree and the only device that can originate configuration transactions. It enumerates the tree, assigns bus numbers, sizes and programs every Base Address Register, and owns the system memory map. An Endpoint is a leaf that answers configuration reads and waits to be told where it lives. An SSD is always an Endpoint; a CPU chipset is normally the Root Complex. On a board with no processor, the FPGA has to take the root role itself.

Does an FPGA need to be the Root Complex to talk to an NVMe SSD?

Yes. An NVMe SSD is a PCIe Endpoint and will never enumerate anything upstream of it. Something has to release the drive from reset, train the link, read its configuration space, program its BARs, enable bus mastering and set up MSI-X before a single NVMe admin command can be issued. If there is no processor in the design, that root-side work has to exist in logic, which is what an NVMe host IP core provides alongside the NVMe command set itself.

Which PCIe generation can my FPGA actually reach?

Whichever one the hard block in that specific device supports. PCIe is hardened silicon next to the transceivers, so the generation is a property of the part number and speed grade, not of the IP you license. As a rough map, Microchip PolarFire tops out at Gen2, AMD UltraScale+ is a Gen3 family with Gen4 on selected devices, Altera Agilex 7 reaches Gen5 with the R-Tile, and AMD Versal Premium reaches Gen5 with Gen6 in the announced Gen2 devices. Always confirm against the vendor documentation for your part.

Why does my PCIe link train at x1 instead of x4?

Almost always because three of the four lanes never reached receiver detect. Look for an unconnected or damaged differential pair, a missing AC coupling capacitor on one lane, or transceiver channels that were not enabled in the design. Read the negotiated link width and the LTSSM state from the link status registers before touching the protocol layers — the specification handles lane reversal and polarity inversion on its own, so those are rarely the explanation.

How many MBytes per second does a PCIe Gen4 x4 link give me?

That is a measurement, not a calculation. PCI-SIG defines Gen4 as 16 GT/s per lane with 128b/130b encoding, and everything after that depends on Max Payload Size, Max Read Request Size, flow-control credits, how deep your read pipeline runs and how the SSD behaves under sustained write. We do not publish a MBytes-per-second figure for a core on a generation we have not measured; the only sustained throughput Logic Design Solutions has published is roughly 500 MBytes per second over a single 6 Gbit SATA link, announced in 2019.

Can the same FPGA be both a Root Complex and an Endpoint?

Yes, if it has enough hard blocks. Many devices carry more than one PCIe hard block, so one can be configured as a root port driving an SSD while another sits as an endpoint facing a host server. That is the usual shape of a recorder card that captures to local NVMe storage and is also read back over the host bus. What a single hard block cannot do is be both at once, so the count of PCIe blocks on the device becomes a selection criterion.

  1. PCI-SIG — PCI Express Base Specification: normative source for topology, configuration space, TLP formats, flow control, LTSSM and the per-lane transfer rate of every generation.
  2. PCI-SIG — PCI Express technology overview: the standards body that defines PCI Express and publishes the generation roadmap including the PAM4 and FLIT-mode change at Gen6.
  3. NVM Express, Inc. — NVM Express specifications: defines the controller registers, doorbells, queue model and MSI-X usage that sit in PCIe BAR space.
  4. AMD — Versal Premium series adaptive SoCs: vendor documentation for the hard PCIe blocks and integrated DMA on Versal Premium.
  5. AMD — FPGAs and adaptive SoCs product family: vendor documentation for the PCIe integrated blocks across the UltraScale+ and Versal families.
  6. Altera — Agilex FPGAs and SoCs: vendor documentation for the R-Tile and P-Tile hard PCIe blocks and their Avalon interfaces.
  7. Microchip — PolarFire FPGAs: vendor documentation for the PolarFire embedded PCIe hard block, configurable as root port or endpoint with AXI4 interfaces.
  8. ServeTheHome — AMD Versal Premium Gen2 at Hot Chips 2026, 25 August 2026: reported PCIe Gen1 to Gen6 SerDes with Gen7 signalling and CXL 4.0 taken into the programmable logic.