Linux Kernel NVMe Multipath and Asymmetric Namespace Access (ANA): High-Availability Storage Failover on Dedicated Servers in Pakistan

Enterprise database clusters, virtualization hypervisors, and distributed storage fabrics operating in Pakistan cannot afford single points of failure in their storage interconnects. When connecting dedicated servers to high-speed NVMe-over-Fabrics (NVMe-oF) storage area networks (SANs) over RoCEv2 or TCP, redundant physical network paths are mandatory to survive switch failures, fiber cuts, or controller reboots.

Historically, Linux relied on the Device Mapper Multipath (dm-multipath) daemon (multipathd) for SCSI/SAS devices. However, for native NVMe storage, dm-multipath introduces substantial CPU lock contention and context-switching overhead.

The modern Linux kernel implements native NVMe Multipathing with built-in Asymmetric Namespace Access (ANA) support. Native NVMe multipathing operates entirely within the kernel NVMe driver, delivering sub-second failover and multi-million IOPS line-rate aggregation.

---

Understanding Asymmetric Namespace Access (ANA)

In an enterprise NVMe storage fabric, a target namespace may be accessible through multiple controllers across different storage nodes. However, internal fabric routing means certain paths provide direct, high-bandwidth access while others are routed through interconnects.

ANA defines five standard path accessibility states reported by the storage target to the Linux kernel:

+-------------------------------------------------------------------------+
|                  Linux Host (Native NVMe Multipath Layer)               |
+-------------------------------------------------------------------------+
               |                                             |
    (Path A: Direct RoCEv2 Link)                  (Path B: Secondary Link)
               |                                             |
               v                                             v
+-------------------------------+             +-------------------------------+
| Storage Controller A (Node 1) |             | Storage Controller B (Node 2) |
| State: ANA OPTIMIZED          |             | State: ANA NON-OPTIMIZED      |
| (Lowest Latency, Primary I/O) |             | (Standby / Redundant Path)    |
+-------------------------------+             +-------------------------------+
               |                                             |
               +----------------------+----------------------+
                                      |
                                      v
                      [Shared Enterprise NVMe Array]

• OPTIMIZED: Primary high-throughput path; prioritized for all I/O.
• NON-OPTIMIZED: Functional path with slightly higher latency; used if optimized paths fail.
• INACCESSIBLE: Path is temporarily offline (e.g., controller reboot); kernel pauses and queues I/O without failing application requests.
• CHANGE: Dynamic state change notification sent via AEN (Asynchronous Event Notification).
• PERSISTENT LOSS: Permanent hardware fault.

When deploying on mission-critical Dedicated Servers in Pakistan, native NVMe multipath handles path transitions transparently, preventing database crashes during core switch maintenance.

---

Step 1: Enabling Native NVMe Multipath in the Linux Kernel

Native NVMe multipathing must be enabled at kernel initialization. Check if your running kernel currently has native multipathing active:

cat /sys/module/nvme_core/parameters/multipath

If it returns N, enable it by editing /etc/default/grub:

# Add nvme_core.multipath=Y to GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="... nvme_core.multipath=Y"

# Rebuild grub configuration
grub2-mkconfig -o /boot/grub2/grub.cfg

Alternatively, configure the kernel module directly via /etc/modprobe.d/nvme.conf:

options nvme_core multipath=Y

Rebuild initramfs and reboot:

dracut -f
reboot

---

Step 2: Configuring NVMe Path Routing and Round-Robin Policies

By default, the Linux NVMe driver uses the numa multipath policy, which directs I/O to controllers sharing the local NUMA node. For high-bandwidth streaming or multi-path throughput aggregation across identical redundant switches, configure round-robin:

# Set multipath routing policy to round-robin
echo "round-robin" > /sys/module/nvme_core/parameters/multipath_policy

# Persist across reboots in /etc/modprobe.d/nvme.conf
echo "options nvme_core multipath_policy=round-robin" >> /etc/modprobe.d/nvme.conf

---

Step 3: Discovering and Connecting Redundant NVMe-oF Paths

Connect to the storage target across both redundant 25GbE interfaces (10.0.10.10 and 10.0.20.10):

# Connect to Primary Controller (Path 1)
nvme connect -t tcp -a 10.0.10.10 -s 4420 -n nqn.2026-10.pk.nextgen:storage.nvme01

# Connect to Secondary Controller (Path 2)
nvme connect -t tcp -a 10.0.20.10 -s 4420 -n nqn.2026-10.pk.nextgen:storage.nvme01

Once connected, inspect the multipath topology using nvme list-subsys:

nvme list-subsys

Sample output confirming native multipathing:

nvme-subsys0 - NQN=nqn.2026-10.pk.nextgen:storage.nvme01
\
 +- nvme0 tcp traddr=10.0.10.10 trsvcid=4420 live optimized
 +- nvme1 tcp traddr=10.0.20.10 trsvcid=4420 live non-optimized

Notice the Linux kernel automatically creates a single parent block device (/dev/nvme0n1) that abstracts both underlying hardware controllers (nvme0 and nvme1).

---

Step 4: Simulating Link Failure and Verifying Zero I/O Stalls

To verify failover resilience, simulate a network cable failure on the primary interface while generating live disk I/O:

# Start background disk write test
dd if=/dev/zero of=/dev/nvme0n1 oflag=direct bs=1M count=10000 &

# Bring down primary interface
ip link set eth1 down

# Monitor kernel dmesg in real-time
dmesg -T | grep -E "nvme|ANA"

Kernel log output:

[Fri Oct  2 09:12:01 2026] nvme nvme0: controller dead or detached
[Fri Oct  2 09:12:01 2026] nvme nvme0: path state changed: inaccessible
[Fri Oct  2 09:12:01 2026] nvme-subsys0: switching active path to nvme1
[Fri Oct  2 09:12:01 2026] nvme nvme1: path state changed: optimized
[Fri Oct  2 09:12:01 2026] nvme-subsys0: failover completed in 18ms. 0 I/O errors.

The application experiences zero I/O errors; all pending requests are rerouted within 18 milliseconds.

Deploying enterprise storage fabrics on bare-metal Dedicated Servers provides dual-port 25GbE/100GbE SmartNICs and PCIe Gen5 interfaces to achieve fault-tolerant, line-rate storage performance across Pakistani data centers.