Skip to content

L8 · Node Interconnect#

conceptstub

What it is#

The links inside a server or rack: NVLink and NVSwitch between GPUs, UALink as an open alternative, and the PCIe paths and NUMA placement that connect GPUs, network cards and CPUs.

Why it matters#

The fastest communication happens inside the NVLink domain, so the chattiest parallelism (tensor parallel) is placed there. PCIe and NUMA placement decide whether a GPU reaches its network card at full speed; a GPU and NIC on different CPU sockets can lose a large share of network bandwidth.

In this module#

Intra-node GPU-to-GPU and GPU-to-NIC paths.

Planned content

  • NVLink and NVSwitch generations, NVL72 domains
  • PCIe switches, NUMA, and NIC/GPU affinity
  • How Kubernetes sees topology: GFD labels, Topology Manager, DRA