Building AI Back-End Fabrics: Tyler Conrad at CHI-NOG 13

Doyle had just wrapped his automation-framework talk, and Tyler Conrad was up next. Principal engineer at Arista, Austin, fifteen years split between customer-side and vendor-side. He plugged in and dove right in, “I’m Tyler Conrad with Arista. I build AI networks. Let’s talk about it.”

No agenda, no thesis slide, that was the whole preamble. The next half hour was the most concrete “this is how you actually build the thing” talk I saw at CHI-NOG 13.

I’ve been writing about AI networking for a few months, mostly through the NFD40 vendor coverage. That series settled the conceptual side. Scale-up, scale-out, and scale-across are three different problems. Fine. But I came out of NFD40 with a clear what and a much softer how. Conrad’s talk filled in the how.

Conrad works for Arista, so there’s vendor framing baked into any principal-engineer talk like this. The technical content mostly stands on its own and matches what I’ve heard from operators in the hallway who don’t sell switches.

Front-end and back-end aren’t the same network

An AI host bifurcates into two physically distinct networks. The front-end looks like the data center networks you’ve already built. Services, VXLAN, the host’s path to storage, the internet, the rest of the enterprise. The back-end is purpose-built for GPU-to-GPU collective traffic, the coordinated all-at-once exchanges that training runs on, and basically nothing else.

The line Conrad delivered, that I want every architect reading this to internalize. “The back-end is not one that you want to try to brownfield.” Don’t graft it onto your existing fabric. Don’t run it across shared spines, build it from the ground up.

The reason is the back-end’s zero-oversubscription requirement. Combine that with a modern GPU saturating a 400G NIC, or an 800G NIC with 1.6T coming, each GPU on its own NIC, eight per host, and that host can burst eight times 800G the second a collective fires. Try to absorb that on an oversubscribed enterprise spine and your front-end goes down with it.

And each of those NICs is its own discrete endpoint. You don’t get to consolidate them, no MLAG to a single bond, every NIC is a first-class fabric attachment.

How big you build is set by GPU count

Conrad’s topology decision tree is the cleanest version of this I’ve seen anyone draw. Worth memorizing.

1 GPU server. No back-end needed. The scale-up fabric inside the chassis handles everything. Build a front-end and you’re done.

2 hosts. Now you need a back-end. Two hosts can’t communicate over scale-up. The moment you want collectives to span chassis, the back-end is in your design.

Single leaf (chassis). A modular chassis as a single leaf gives you roughly 1,000 GPUs. The sweet spot for single-tier. If you’re under a thousand GPUs, the chassis is enough.

Two-tier leaf-spine with 64x800G fixed silicon. Meat of the market. With 400-gig generation GPUs on current 64x800G silicon, you get to roughly 8,000 GPUs in a clean leaf-spine. If you’re not hyperscaling, this is where you live.

Hybrid chassis-as-spine plus fixed leaves. Modular chassis as spine, fixed leaves below. The chassis radix stretches to about 73,000 GPUs.

Three-tier all-chassis. Modular at every tier. Just under 663,000 GPUs. Hyperscaler territory.

Three-tier brings a different cost. Every link down to a GPU usually has an equivalent uplink, leaf to spine. Carry that ratio between spine and superspine and your link count roughly doubles. Three-tier is expensive in cabling, not just chassis.

Rails, and why rail-isolated is effectively dead

Rails organize the back-end fabric. Each GPU on a host attaches to a specific position (rail 0 through rail 7 for an 8-GPU host), and the fabric is built so a rail-N NIC always connects to a rail-N leaf. Same rail, same leaf, across every host.

Two flavors. Rail-isolated means each rail is a completely separate leaf-spine cluster. Eight independent fabrics for an 8-rail design. Big scaling wins because you don’t sacrifice half the leaf ports on inter-rail uplinks. Rail-optimized means the rails share a common spine layer, so traffic can cross between rails when it needs to.

Conrad hasn’t seen many rail-isolated deployments in the field. They exist. They’re just rare. Most operators run rail-optimized.

The reason is failure recovery. In rail-isolated, if a link fails, or a GPU fails, or the scheduler picks GPUs without knowing exactly what rail they’re on (it makes assumptions through NCCL (NVIDIA’s collective library) and may guess wrong), you don’t have a clean way to route around it. Rail-optimized gives you the spine as an escape hatch. Slightly less scale efficient. Operationally much more forgiving.

If you’re designing rail-isolated right now because the math looks cleaner on paper, you’re fighting yesterday’s battle.

PXN, and the shortcut through scale-up

This is the slide that got me to open my notebook.

NVIDIA and AMD both ship a feature called PXN. PCI-E Extended Networking, per Conrad’s slide. The scale-up fabric inside the chassis isn’t just for collectives between local GPUs. It can be a shortcut between rails on the same remote host.

In his example, GPU 1 sits on rail 1 of the local host and wants to talk to GPU 6 on rail 6 of a remote host. Normal path. Up from local GPU 1’s NIC to local leaf, across the spine, down to remote leaf on rail 6, into GPU 6. Three hops of latency.

PXN path. GPU 1 sends to its local rail 1 NIC, the local scale-up fabric hops sideways to GPU 6’s local NIC (which is on rail 6 within the local host), and that NIC sends directly across to the remote rail 6 leaf and into the remote GPU 6. The scale-up fabric does one cross-rail hop locally so the back-end fabric never traverses the spine.

Scale-up fabrics are very high bandwidth and very low latency, so using them as a same-host shortcut between rails is essentially free.

It needs PXN-aware NICs and the collective library has to know how to use it. But if you’re running NVIDIA or AMD and you’re building rail-optimized, this is on by default in most current stacks.

You can’t oversubscribe a back-end

I flagged this earlier as the reason you can’t brownfield a back-end. Here’s the full version. The rule is 1:1. Zero oversubscription.

Every GPU bursts at line rate at the same moment when a collective fires. There’s no statistical multiplexing to take advantage of. The whole point of an all-reduce is that every GPU is talking simultaneously, every link is hot, and the collective doesn’t complete until the slowest path completes. Any oversubscription anywhere means the slowest path runs through your bottleneck and you’ve paid for compute that’s sitting idle.

A 64x800G switch is therefore typically split 32 downlinks, 32 uplinks. Half and half. Period.

Here is where AI fabric design breaks every habit from cloud networking. Twenty years of “we can hash, we can statistically multiplex, we can absorb bursts in queue” doesn’t apply. The bursts aren’t statistical. They’re coordinated. Every collective lights every link at the same instant.

LPO, and the 256 kilowatts hiding in your optics bill

Linear pluggable optics (LPO) deserve more attention than they got at NFD40, and Conrad gave them the moment.

The DSP (digital signal processor) inside a typical pluggable optic is power-hungry. Move it from the optic into the switch silicon and you cut per-optic power roughly in half. He put a number on it, ~16W per optic with LPO, versus ~32W with the DSP onboard. He noted in passing that 16W is roughly what your iPhone pulls off the wall.

Sounds small. Run the math at scale. In an 8K-GPU cluster, you have around 16,000 optics. At 16W saved per optic, that’s 256 kilowatts. Cluster total. Before HVAC. Before PDU overhead. Just optics.

Layer cooling on top. Every watt the gear draws pulls roughly another 30% in cooling and overhead (PUE 1.3, if you speak data-center facilities), so cutting 256 kW of optics load saves about another 75 kW. Call it 330 kilowatts of facility power saved per 8,000 GPUs. That number actually moves the financial model.

Vendors don’t lead with this because it doesn’t fit cleanly into a buy decision matrix. If you’re sizing a build-out for the next decade of GPU growth and you’re not asking your optics vendor about LPO, you’re going to wear it. The silicon for host-side DSP is already shipping from Broadcom and NVIDIA. The optic side is following.

The ECMP elephant flow problem

You knew this section was coming. ECMP hashes flows to uplinks. AI traffic is a small number of enormous flows. 400G flows hash, sometimes badly, onto the same 400G uplink.

I covered this in the NFD40 wrap-up so I won’t relitigate. Conrad’s coverage was the operator-side version. The two real mitigations.

Dynamic load balancing. Vendor-specific (Arista, NVIDIA, others each have a flavor). Periodically measure per-port utilization and steer arriving flows to the least-loaded link. Sidesteps the hash collision problem at the cost of vendor lock-in on the load-balancing behavior itself.

Packet spraying via QPairs. RDMA (remote direct memory access) breaks the conversation into queue pairs, and you can spray QPairs across multiple spines so any given flow uses many paths. Closer to what Ethernet should have done from the start.

Here’s the catch I want to flag. RDMA is fragile to out-of-order delivery. Conrad’s exact framing. “If it receives packets out of sequence, it’s basically going back to the end problem. It has to wait until it receives the next packet and it will drop everything after that until it does.” Packet spraying requires a NIC-side wrapper that puts packets back in order before handing them to the application. NVIDIA, AMD, and a few merchant NIC vendors ship this. The NIC has to participate.

That’s the implementation detail that doesn’t show up in the architecture diagrams and absolutely shows up at 2 AM when something’s slow and you can’t explain why. The wrapper is a non-optional dependency for packet spraying.

SRv6 microSID for source-based routing

Conrad also pulled SRv6 microSID out for a closer look.

Quick primer, since SRv6 isn’t everyday kit for most of us. It’s segment routing carried over IPv6. Instead of each switch making its own hop-by-hop forwarding decision, the source writes the whole path into the packet as a list of waypoints, and every hop just follows the next one. That’s the lever for the problem the ECMP section kept running into, getting traffic onto specific, deterministic paths, except here the source picks the path outright instead of hoping a hash spreads the load.

This isn’t full SRv6. It’s microSID (sometimes uSID). The mechanism. The source crams a list of node identifiers into the destination IPv6 address itself. Each hop reads the leftmost identifier, strips it off via a shift-left operation that rewrites the IPv6 address in place, and forwards to whatever that identifier resolves to. The packet carries its own list of waypoints in its header.

Why this works in an AI back-end is interesting. You get deterministic source routing without an SRv6 controller, which has historically been the friction point keeping operators away from SRv6. The source decides the path. The network executes.

Conrad pointed at a Containerlab SRv6 topology for anyone who wants to play with it.

PFC and ECN belong together

RDMA doesn’t tolerate loss, so the back-end has to be lossless. Two mechanisms handle it, PFC and ECN.

PFC. Priority Flow Control. When a switch buffer fills, it sends a pause frame upstream telling the previous hop to stop sending for the affected traffic class. Hop by hop. The catch with PFC… the pauses cascade. A pause at one switch builds back through the topology and starts affecting flows that have nothing to do with the original congestion. Pause storms are a real failure mode in PFC-only fabrics.

ECN. Explicit Congestion Notification. End to end. The congested switch sets the ECN bit in the IP header rather than pausing upstream. The receiver tells the sender. The sender slows down at the application/transport layer. Slowing happens at the source, not at every hop.

Conrad’s combination. PFC absorbs the quick bursts (microsecond-scale) so the network doesn’t drop. ECN handles sustained congestion (millisecond-scale) by telling the sender to back off. You need both. PFC alone is dangerous. ECN alone is too slow to prevent loss.

If your back-end isn’t running both, it’s not lossless. It just hasn’t found the failure mode yet.

OpenAI’s MRC contribution

Worth a paragraph because it’s the most interesting deployed assembly of these primitives.

OpenAI contributed MRC (Multipath Reliable Connection) to OCP, and it made the rounds a few weeks before CHI-NOG. It stacks multi-plane topology, packet spraying across planes, ECN for congestion notification, and SRv6 microSID for deterministic path selection, then adds a novel path-health monitor.

The monitor is the part I like. The GPU host sends an SRv6 probe whose path list ends with its own address. The network executes the path. The last hop loops back to the originating host. No control plane. No SRv6 controller. The host sees its own probe come back, latency stamped, and knows that specific path is healthy. Repeat every millisecond across as many paths as the host wants to monitor.

It works precisely because SRv6 made the network dumb enough to do what the source told it to do.

And then there’s the Ethernet vs InfiniBand claim

During Q&A, somebody asked Conrad whether Ethernet was going to win the back-end.

Ethernet has already surpassed InfiniBand in market-share revenue for AI cluster spend. He was upfront that these are analyst estimates, not hard numbers, but said the trend line isn’t ambiguous.

Consequential claim, and yes, Conrad works for the vendor that just won that fight, so weigh it accordingly. For what it’s worth, the public analyst numbers back him up. Dell’Oro reported Ethernet overtaking InfiniBand in AI back-end switch revenue in 2025. Mark it as a real industry data point framed by an Arista engineer, not an Arista claim.

If true, the next front is scale-up, the proprietary fabric inside the chassis. NVIDIA owns scale-up right now via NVLink. AMD has their own answer. Arista, Broadcom, and the rest of the merchant ecosystem are pushing UALink (Ultra Accelerator Link) and ESUN (Ethernet for Scale-Up Networking) to standardize the chassis interior the same way they standardized the chassis exterior. Whether that fight goes the same way Ethernet did against InfiniBand is the next decade’s question. Conrad didn’t say that on stage. But it’s the implication of the data point he dropped.

The takeaway

If you read the NFD40 wrap-up, the framing was that AI networking is a different category of design problem. A category with its own math and its own failure modes.

Conrad’s CHI-NOG talk filled in the workshop manual. Build the back-end fresh. Use rail-optimized unless you have a hyperscaler-grade reason not to. Run 1:1 oversubscription. Get PXN on, get packet spraying with a NIC-side reordering wrapper, decide which load balancing flavor your silicon supports, run PFC and ECN together. Pick LPO when the silicon is ready, because the power math shows up in facility planning. Watch SRv6 microSID, because deterministic source routing without a controller will mature into something more operators reach for.

Recording’s below. Whether or not you run Arista, the back-end mechanics here are vendor-agnostic even when the silicon examples aren’t, so it’s worth the half hour.

Disclosure: I attended CHI-NOG 13 in Chicago. CHI-NOG didn’t comp my registration or travel. Arista didn’t buy me anything. The opinions here are mine. For more, please read my full disclaimer.

Links and references

• What NFD40 Taught Me About AI Networking, the conceptual counterpart to this post
• CHI-NOG 13, conference site
• OCP MRC spec, the OpenAI-contributed spec that made the SRv6 microSID approach concrete