How to Fix 'Read-Only File System' Error on Linux: XFS Repair, Emergency Remount & Storage Diagnostics in Pakistan

You SSH into your production web server in Lahore or Karachi, attempt to edit an Nginx virtual host, create a MySQL database, or write a log file, and your shell abruptly returns:

touch: cannot touch 'test.txt': Read-only file system
Error writing /etc/nginx/nginx.conf: Read-only file system

Moments later, web services collapse across the server: Nginx throws 500 Internal Server Error, MySQL crashes because it cannot append to redo logs or transaction state files, and cPanel throws licensing validation panics.

This catastrophic behavior is an intentional Linux kernel self-defense mechanism known as errors=remount-ro. When the Linux kernel storage driver encounters an uncorrectable I/O block error, metadata journal corruption, or excessive hardware timeout from the underlying block device, it immediately cuts write access to prevent silent data corruption across remaining uncorrupted partitions.

While running your infrastructure on enterprise-grade Cloud VPS instances or bare-metal Dedicated Servers with redundant NVMe RAID arrays minimizes physical storage dropouts, sysadmins must know how to quickly diagnose root causes and restore write capability safely without losing production data.

In this deep-dive diagnostic guide, we examine the kernel panic trace using dmesg, explain how to execute emergency read-write remounts, detail safe offline repair procedures using xfs_repair and fsck.ext4, and inspect NVMe SMART metrics to prevent recurrent hardware failures.

---

1. Inspecting the Kernel Ring Buffer: Finding Root Cause

Never blindly reboot or attempt a blind remount before understanding why the kernel transitioned the filesystem into read-only mode.

Open your terminal and inspect dmesg or /var/log/messages:

# Filter recent kernel ring buffer messages for filesystem errors
dmesg -T | grep -E -i "read-only|remount|error|xfs|ext4|io|abort|corrupt" | tail -n 30

Typical error patterns you will encounter:

Scenario A: XFS Metadata Journal Corruption (AlmaLinux / Rocky / CentOS / RHEL)

[Fri Oct 10 04:12:08 2026] XFS (nvme0n1p2): Metadata corruption detected at xfs_inode_buf_verify+0x180/0x1e0 [xfs]
[Fri Oct 10 04:12:08 2026] XFS (nvme0n1p2): Filesystem has been shut down due to log error (0x2).
[Fri Oct 10 04:12:08 2026] XFS (nvme0n1p2): Please unmount the filesystem and run xfs_repair to resolve.

Scenario B: Ext4 Block Device Timeout / I/O Error (Ubuntu / Debian)

[Fri Oct 10 04:12:08 2026] EXT4-fs error (device sda1): ext4_lookup:1842: inode #262145: comm nginx: deleted inode referenced: 262148
[Fri Oct 10 04:12:08 2026] Aborting journal on device sda1-8.
[Fri Oct 10 04:12:08 2026] EXT4-fs (sda1): Remounting filesystem read-only

---

2. Emergency Assessment: Ext4 vs XFS

The repair procedure differs fundamentally based on the underlying filesystem format. Verify which filesystem your affected partition uses:

# Check filesystem types and mount options
df -Th
lsblk -f

Filesystem Type	Default Behavior on Kernel Error	Repair Tool	Online Repair Supported?
Ext4	Remounts read-only (errors=remount-ro)	fsck.ext4 / e2fsck	NO (Must be unmounted or RO)
XFS	Shuts down filesystem (fs shutdown)	xfs_repair	NO (Never run on mounted fs)
Btrfs	Forces read-only transactions	btrfs check	NO (Requires unmounted state)

---

3. Quick Temporary Mitigation: Attempting Emergency Remount

If the read-only transition was triggered by a transient storage controller hiccup (such as a temporary SAN hypervisor failover on virtualized hosting) rather than physical block corruption, you can test if the kernel permits an emergency read-write remount:

# Attempt to remount the root filesystem in read-write mode
mount -o remount,rw /

# Verify write capability
touch /root/rw_test && rm -f /root/rw_test && echo "✅ Filesystem writable!"

If the command succeeds and dmesg remains clean, monitor I/O carefully. If the command fails with:

mount: /: can't read superblock on /dev/nvme0n1p2 or write protected

you must proceed immediately to offline filesystem repair.

---

4. Repairing Corrupted XFS Partitions with xfs_repair

Unlike Ext4, XFS maintains strict metadata integrity structures that cannot be repaired while the filesystem is mounted read-write or mounted at all.

Step 4.1: Unmount the Affected Volume

If the affected partition is a secondary volume (e.g., /home, /var, or /backup):

# 1. Stop all services accessing the partition
systemctl stop nginx mariadb httpd

# 2. Terminate any clinging processes
fuser -kvm /home

# 3. Unmount the filesystem cleanly
umount /home

Step 4.2: Execute xfs_repair

Run the XFS repair utility against the raw block partition:

# Inspect and repair the unmounted partition
xfs_repair /dev/nvme0n1p3

If the log is damaged and the normal repair halts requesting log replay:

# FORCE zeroing out a corrupted transaction log (Emergency repair)
xfs_repair -L /dev/nvme0n1p3

Warning on -L flag: Zeroing the log (-L) discards uncommitted journal metadata from the moment of the crash. Use this only when a standard xfs_repair refuses to complete.

---

5. Repairing Corrupted Ext4 Partitions with fsck

For Ubuntu and Debian systems using Ext4:

# 1. Unmount the target volume
umount /dev/sdb1

# 2. Run e2fsck with non-destructive read test and automatic safe fixes
e2fsck -f -y -v /dev/sdb1

Repairing the Root (/) Partition via Emergency Boot

If the root filesystem itself is locked:

1. Schedule a forced check on next boot:

   # Create forcefsck trigger or tune filesystem flags
   tune2fs -c 1 /dev/sda1

2. Alternatively, access your VPS console / IPMI KVM, reboot, and at the GRUB menu, append fsck.mode=force fsck.repair=yes to the kernel arguments.
3. Once in rescue mode, run fsck -y /dev/sda1 before mounting.

---

6. Verifying Hardware Storage Health (SMART Diagnostics)

Filesystem corruption is frequently an early warning signal of impending physical SSD/NVMe drive failure. Once write access is restored, check hardware SMART telemetry:

# For NVMe Drives: Install nvme-cli and check device health
apt-get install -y nvme-cli || dnf install -y nvme-cli
nvme smart-log /dev/nvme0

# For SATA / SAS SSDs: Check SMART attributes
smartctl -H -A /dev/sda

Key indicators of hardware failure:

• media_errors > 0: Unrecoverable data integrity errors encountered by the NAND controller.
• critical_warning > 0: Spare blocks exhausted or temperature throttling breached.
• available_spare < 10%: Drive has exhausted its reserve blocks.

If media errors appear in SMART logs, replace the drive immediately.

---

7. Zero-Downtime Storage Architecture for Enterprise Workloads

Filesystem lockdowns halt revenue-generating applications instantly. Prevent single-point storage failures by hosting your databases and critical workloads on high-endurance, monitored enterprise hardware.

Explore our related infrastructure tutorials:

• Fix MySQL InnoDB Table Corruption Masterclass
• PostgreSQL Performance Tuning: Shared Buffers & Memory
• Linux eBPF Line-Rate DDoS Mitigation in Pakistan

For organizations requiring enterprise Samsung/Micron NVMe storage with hardware RAID-10, out-of-band IPMI KVM management, and 24/7 on-site datacenter engineers in Lahore, deploy on Dedicated Servers in Pakistan.