InfiniBand Deep Dive: Networking for AI focused Datacentres

Use an analogy of road evolution to illustrate how InfiniBand addresses the need for high speed, low latency, lossless data transmission in AI-focused datacentres, contrasting with Ethernet's evolution.

Connect cpu-based servers, GPU-based servers, and storage on an InfiniBand network, scale with switches, and bridge to Ethernet via a gateway for redundancy.

Discover InfiniBand architecture layers: upper workload interfaces like NVMe over fabrics, GPU direct RDMA, and IP over InfiniBand, plus transport, network, link, and physical layers enabling zero-copy end-to-end HPC communication.

Explore the InfiniBand software and hardware stack, from host channel adapters and cabling to InfiniBand switches, and how the data plane and subnet manager coordinate communication.

Explore InfiniBand physical connectivity across compute, GPU, and storage nodes from various vendors using HCA host channel adapters or embedded adapters, connecting through an InfiniBand switch and cables.

Explore host channel adapters in InfiniBand networks, including CPU offload and single-port or multiport SCAs. Learn about ConnectX from NVIDIA, supporting InfiniBand and Ethernet with offload capabilities.

Set up two-node InfiniBand connectivity on Ubuntu using either a copper DAC cable or an InfiniBand switch, install the OFED or RDMA stack, and validate with the IBFDV info command.

Verify the InfiniBand card with idverb and devinfo, inspect port status and GUIDs, and note that the subnet manager is inactive with no LID.

Explore a hands-on InfiniBand switch setup, connecting two devices to port 2 and port 3, observing LED indicators and basic OpenSM readiness in a datacenter networking context.

Explain how global unique identifiers, including system image GUIDs and node and port GUIDs, identify hardware and guide the subnet manager in LED assignment within an InfiniBand system.

Explore how ID ping provides InfiniBand connectivity tests between nodes using a client-server model (IDPing-S and IDPing-N), measuring latency and aiding MPI, CCL, and RDMA debugging.

Discover and map your InfiniBand fabric using IBNet Discover to visualize topology, including nodes, SCAs, switches, ports, and LIDs, with a practical example of two servers connected to a switch.

Explore InfiniBand utilities IB Link Info and IB TraceRT to view link level connectivity, topology, and path traces across ports, switches, and channel adapters.

Explore InfiniBand utilities for fabric discovery and hardware validation, including ib host, ib switches, ib nodes, and ib virb device info.

Discover how InfiniBand uses remote direct memory access to move data directly between devices and system memory, bypassing the CPU, and increasing data transfer speeds.

Discover how remote DMA (RDMA) enables direct memory access across hosts over InfiniBand, bypassing the CPU to boost throughput and reduce network latency in high-performance computing (HPC) environments.

Analyze how traditional data transfer uses sockets, TCP/IP, NIC, and the operating system, causing CPU overhead and high latency. See how RDMA solves these problems by reducing memory copies.

Accelerate data transfer with RDMA by bypassing the tcp/ip stack, eliminating cpu involvement and copies, and using RDMA verbs with direct access to the network interface card.

Discover how RDMA enables zero copy transfer by allowing the source to write directly into destination memory using RDMA verbs, bypassing CPU and kernel copies for faster data transfers.

Learn how RDMA verbs are the low-level API that lets applications send, read, and write data directly to remote memory via the NIC, and how InfiniBand features enable native RDMA.

Explore the queue pair (QP) as the fundamental InfiniBand communication endpoint, detailing how RDMA verbs create QPs, use send and receive queues, DMA data, and completion queues.

Explore InfiniBand lossless, credit-based flow control, where senders transmit only when receivers are ready, leveraging queue pairs and memory registration to enable rdma-based AI datacenter applications.

NVIDIA extends RDMA with GPU direct RDMA, enabling direct GPU-to-GPU memory access across nodes, bypassing CPU and OS, using ROCE or InfiniBand with NVIDIA drivers.

Explore GPU direct storage, which bypasses the CPU and system memory to transfer data from NVMe storage directly to the GPU, reducing IO bottlenecks during training.

Clarify GPU direct RDMA versus GPU direct storage by highlighting their scopes, data paths, low latency networking, and high bandwidth storage pipelines for AI and HPC.