InfiniBand Deep Dive: Networking for AI focused Datacentres

InfiniBand powers modern data centers as the backbone of faster networks, addressing data growth, distributed computing, AI/ML training, real-time low-latency needs, and full compute utilization.

Use an analogy of road evolution to illustrate how InfiniBand addresses the need for high speed, low latency, lossless data transmission in AI-focused datacentres, contrasting with Ethernet's evolution.

Explore the evolution of InfiniBand from 10 Gbps SDR to 800 Gbps XDR and the upcoming GDR, highlighting how modern data centers leverage NDR and XDR for high performance.

Connect cpu-based servers, GPU-based servers, and storage on an InfiniBand network, scale with switches, and bridge to Ethernet via a gateway for redundancy.

Relate InfiniBand's architectural layers to a package delivery system, mapping workloads like video or AI inferences to packages, and illustrate delivery via courier services, GPS, traffic rules, and road infrastructure.

Discover InfiniBand architecture layers: upper workload interfaces like NVMe over fabrics, GPU direct RDMA, and IP over InfiniBand, plus transport, network, link, and physical layers enabling zero-copy end-to-end HPC communication.

Explore the InfiniBand software and hardware stack, from host channel adapters and cabling to InfiniBand switches, and how the data plane and subnet manager coordinate communication.

Explore InfiniBand physical connectivity across compute, GPU, and storage nodes from various vendors using HCA host channel adapters or embedded adapters, connecting through an InfiniBand switch and cables.

Explore host channel adapters in InfiniBand networks, including CPU offload and single-port or multiport SCAs. Learn about ConnectX from NVIDIA, supporting InfiniBand and Ethernet with offload capabilities.

Compare InfiniBand cabling options, including copper DAC and optical AOC, noting range, power, cost, and EMI implications, and demonstrate how to connect InfiniBand devices across a data center.

Set up two-node InfiniBand connectivity on Ubuntu using either a copper DAC cable or an InfiniBand switch, install the OFED or RDMA stack, and validate with the IBFDV info command.

Configure the first InfiniBand node by verifying physical card installation and detection, then install the MLNX or NVIDIA OFED driver on Linux and load Mellanox modules.

Explore the InfiniBand module stack that enables communication between applications like MPI, NCCL, and IBWorks, through IBUWorks, IBCore, MLX4IB, and MLXCore, and learn how to troubleshoot and test the network.

Verify the InfiniBand card with idverb and devinfo, inspect port status and GUIDs, and note that the subnet manager is inactive with no LID.

Connected two systems with a physical cable and verified the metanox card and MLX module using ibv dev device info, before subnet manager activation configures the fabric.

Discover, configure, and maintain the InfiniBand fabric with a subnet manager—the brain that assigns identifiers, discovers topologies, builds routing tables, avoids loops and deadlocks, and balances paths for optimal performance.

Learn to install OpenSM, load rdma-core utilities, start the subnet manager as a service, and validate a two-node InfiniBand setup.

Explore a hands-on InfiniBand switch setup, connecting two devices to port 2 and port 3, observing LED indicators and basic OpenSM readiness in a datacenter networking context.

Learn to inspect InfiniBand interfaces with ibstate and infiniband-diagnose, and interpret adapter details such as PCI location, MT4099 hardware, port status, link speed, base lid, and sm lid.

Explain how global unique identifiers, including system image GUIDs and node and port GUIDs, identify hardware and guide the subnet manager in LED assignment within an InfiniBand system.

Discover how local identifiers (LIDs) enable fast, hardware-level routing in InfiniBand fabrics by mapping 16-bit addresses to ports, decoupling routing from GUIDs, and enabling efficient switch lookups.

Explore how ID ping provides InfiniBand connectivity tests between nodes using a client-server model (IDPing-S and IDPing-N), measuring latency and aiding MPI, CCL, and RDMA debugging.

Discover and map your InfiniBand fabric using IBNet Discover to visualize topology, including nodes, SCAs, switches, ports, and LIDs, with a practical example of two servers connected to a switch.

Explore InfiniBand utilities IB Link Info and IB TraceRT to view link level connectivity, topology, and path traces across ports, switches, and channel adapters.

Explore InfiniBand utilities for fabric discovery and hardware validation, including ib host, ib switches, ib nodes, and ib virb device info.

Discover how InfiniBand uses remote direct memory access to move data directly between devices and system memory, bypassing the CPU, and increasing data transfer speeds.

Discover how remote DMA (RDMA) enables direct memory access across hosts over InfiniBand, bypassing the CPU to boost throughput and reduce network latency in high-performance computing (HPC) environments.

Analyze how traditional data transfer uses sockets, TCP/IP, NIC, and the operating system, causing CPU overhead and high latency. See how RDMA solves these problems by reducing memory copies.

Accelerate data transfer with RDMA by bypassing the tcp/ip stack, eliminating cpu involvement and copies, and using RDMA verbs with direct access to the network interface card.

Discover how RDMA enables zero copy transfer by allowing the source to write directly into destination memory using RDMA verbs, bypassing CPU and kernel copies for faster data transfers.

Learn how RDMA verbs are the low-level API that lets applications send, read, and write data directly to remote memory via the NIC, and how InfiniBand features enable native RDMA.

InfiniBand naturally fits RDMA by providing queue pairs with send and receive queues, hardware offload, memory registration with local and remote keys, and lossless flow control.

Explore the queue pair (QP) as the fundamental InfiniBand communication endpoint, detailing how RDMA verbs create QPs, use send and receive queues, DMA data, and completion queues.

Register memory with the RDMA NIC to enable zero-copy direct memory access, enforcing hardware-level protection with local and remote keys for authorized RDMA access.

InfiniBand offloads transport processing, data movement, and security to the NIC hardware, enabling DMA-based memory-to-memory transfers, hardware RDMA via QPair, and key-based memory protection.

Explore InfiniBand lossless, credit-based flow control, where senders transmit only when receivers are ready, leveraging queue pairs and memory registration to enable rdma-based AI datacenter applications.

Demonstrates RDMA in InfiniBand by teaching how to create a QPair, register memory with local and remote keys, and run bandwidth and latency tests between hosts using IBWriteBW and IBWriteLatency.

NVIDIA extends RDMA with GPU direct RDMA, enabling direct GPU-to-GPU memory access across nodes, bypassing CPU and OS, using ROCE or InfiniBand with NVIDIA drivers.

Explore GPU direct storage, which bypasses the CPU and system memory to transfer data from NVMe storage directly to the GPU, reducing IO bottlenecks during training.

Clarify GPU direct RDMA versus GPU direct storage by highlighting their scopes, data paths, low latency networking, and high bandwidth storage pipelines for AI and HPC.