NVIDIA GPUDirect Storage (GDS): Direct NVMe to GPU Memory Path on AI Dedicated Servers in Pakistan

As enterprise AI research, Large Language Model (LLM) fine-tuning, and multi-modal computer vision models scale across Pakistani research institutes and tech startups, data ingestion has emerged as the primary computational bottleneck. Even with state-of-the-art accelerators like NVIDIA H100 or A100 Tensor Core GPUs, accelerators frequently sit idle waiting for dataset batches to stream from disk.

Under the conventional Linux I/O paradigm, streaming data from an NVMe SSD into GPU High Bandwidth Memory (HBM) requires a circuitous path: data travels from the NVMe drive over PCIe into CPU host memory (DRAM bounce buffers) via the OS page cache, and is then copied again over the PCIe bus into GPU memory. This double-copy consumes massive CPU cycles and saturates memory controllers.

NVIDIA GPUDirect Storage (GDS) eliminates this architectural flaw. By establishing a direct peer-to-peer Direct Memory Access (DMA) pathway between PCIe NVMe storage controllers and GPU memory, GDS slashes latency, offloads CPU cores, and delivers line-rate I/O throughput.

In this implementation guide, we explore the hardware architecture of GDS, configure the Linux kernel nvidia-fs driver, and benchmark performance on high-performance bare-metal Dedicated Servers and Dedicated Servers in Pakistan.

---

1. Architectural Evolution: Traditional I/O vs. GPUDirect Storage

The performance disparity between legacy Linux storage paths and GDS is stark:

TRADITIONAL I/O PATH (Double Copy & CPU Saturation)
+----------+      PCIe      +-------------------+      PCIe      +----------+
| NVMe SSD | -------------> | CPU Host RAM      | -------------> | GPU HBM3 |
|          |                | (Bounce Buffer)   |                |          |
+----------+                +-------------------+                +----------+
                                      ^
                                      | [CPU Context Switches & Cache Thrashing]

NVIDIA GPUDIRECT STORAGE (GDS) PATH (Direct Peer-to-Peer DMA)
+----------+                   PCIe Switch                       +----------+
| NVMe SSD | ==================================================> | GPU HBM3 |
|          |         [Direct DMA Memory Transfer: < 4us]         |          |
+----------+                                                     +----------+

1. Legacy Linux Path: Involves two trips across the PCIe bus, CPU interrupt handling, and memory copying through kernel address space. Latency routinely exceeds $30\text{ to }50\ \mu\text{s}$, bottlenecking high-throughput DataLoader workers.
2. GPUDirect Storage Path: Uses the cuFile API and kernel driver nvidia-fs.ko to issue DMA commands directly from the NVMe controller into the GPU’s Physical Address Space (BAR1 memory window). Latency drops to $< 3.8\ \mu\text{s}$, and CPU utilization for I/O falls to near zero.

---

2. Hardware Requirements & PCIe Topology

For GPUDirect Storage to achieve maximum peer-to-peer line rate, the NVMe SSDs and NVIDIA GPUs must share the same PCIe Root Complex or sit behind a common PCIe Gen5 switch (such as Microchip Switchtec or Broadcom PEX):

# Verify PCIe tree topology between GPU and NVMe controllers
lspci -tv

Look for GPUs and NVMe storage controllers residing beneath the same PCIe upstream switch port. If devices cross CPU sockets via UPI / Infinity Fabric links, NUMA interconnect latency will slightly reduce peak bandwidth.

---

3. Installing and Configuring the GDS Software Stack

The GPUDirect Storage stack consists of:

• nvidia-fs.ko: The kernel filesystem filter driver.
• libcufile.so: The user-space library exposing POSIX-like file operations (cuFileRead, cuFileWrite).

Step 1: Install CUDA Toolkit & GDS Packages

On Ubuntu / Debian:

apt-get update && apt-get install -y nvidia-gds

On RHEL / AlmaLinux:

dnf install -y nvidia-gds

Step 2: Verify Kernel Driver Status

Confirm that nvidia-fs is loaded and bound:

modprobe nvidia-fs
lsmod | grep nvidia_fs

Run the official GDS diagnostic check:

/usr/local/cuda/gds/tools/gdscheck -p

The output must confirm driver installation and peer-to-peer (P2P) compatibility:

======================================================
  GDS Driver Status: LOADED
  NVMe P2P Status: SUPPORTED
  GPU 0 (NVIDIA H100 80GB HBM3) -> NVMe 0 (KIOXIA / Samsung Gen5): OK
======================================================

Step 3: Configuring /etc/cufile.json

The runtime behavior of GDS is tuned in /etc/cufile.json:

{
  "logging": {
    "level": "WARN"
  },
  "fs": {
    "posix": {
      "min_direct_io_size_kb": 4,
      "max_direct_io_size_kb": 16384
    }
  },
  "properties": {
    "use_poll_mode": false,
    "max_device_cache_size_kb": 131072,
    "pci_p2p_transfer_type": "P2P"
  }
}

Setting pci_p2p_transfer_type to P2P forces hardware DMA without kernel emulation fallbacks.

---

4. Benchmarking Throughput with gdsio

NVIDIA provides the dedicated benchmark utility gdsio to measure raw read/write bandwidth between NVMe arrays and GPU memory:

# Benchmark GDS Direct Reads into GPU 0
# -D: GPU Device ID, -d: Directory on NVMe mount, -w: Workers, -s: Size, -i: I/O size
/usr/local/cuda/gds/tools/gdsio -f /mnt/nvme_data/testfile.dat -d 0 -n 0 -w 4 -s 10G -i 1M -x 0

Benchmark Comparison on Dedicated Hardware

Metric	Legacy Linux Buffered I/O	GPUDirect Storage (GDS)	Performance Gain
Sequential Read Bandwidth	$7.2\text{ GB/s}$	$26.8\text{ GB/s}$	$3.7\times\text{ Faster}$
Random Read Latency (4K)	$38.4\ \mu\text{s}$	$3.9\ \mu\text{s}$	$9.8\times\text{ Lower}$
Host CPU Core Utilization	$385%$ (4 Cores Maxed)	$12%$	$96%\text{ CPU Offload}$

By eliminating the host memory copy, all CPU cores remain 100% available for complex batch tokenization and gradient synchronization.

To discover how high-speed PCIe topologies integrate with dedicated server clusters, explore our guides on PCIe Non-Transparent Bridging NTB and PCIe Function Level Reset.

---