MariaDB InnoDB Page Cleaners & Flush List Spikes Prevention on NVMe in Pakistan

Database administrators managing high-volume MariaDB production systems across Pakistan—supporting busy retail POS synchronizations, courier tracking updates, and high-concurrency payment gateways—frequently encounter a perplexing, intermittent performance issue: the database runs smoothly for 20 to 30 minutes, and then suddenly, all queries (even simple SELECT 1 health checks) freeze completely for 3 to 8 seconds. During these freezes, active database threads surge to maximum limits, and web servers throw HTTP 504 Gateway Timeout errors to end-users.

When examining MariaDB’s internal metrics, the cause of these sudden query freezes is known as InnoDB Flush List Freezes (Furious Flushing). In MariaDB, when queries modify data rows, changes are written first to the in-memory Buffer Pool (creating “dirty pages”) and appended to the sequential write-ahead Redo Log.

In background operation, specialized background threads called Page Cleaners are responsible for smoothly and continuously writing dirty pages from the Buffer Pool to physical disk storage. If the number of page cleaner threads is insufficient—or if their scan depth and I/O capacities are calibrated for legacy spinning hard drives—dirty pages accumulate faster than they can be flushed. When the buffer pool fills with dirty pages or the Redo Log runs out of free headroom, MariaDB halts all incoming transactions and triggers an emergency synchronous flush burst that locks the entire database engine.

By deploying on high-IOPS bare-metal Dedicated Servers and properly tuning innodb_page_cleaners, innodb_buffer_pool_instances, innodb_lru_scan_depth, and innodb_io_capacity_max, administrators can achieve uninterrupted, smooth page flushing and eliminate I/O stalls forever.

---

The Anatomy of InnoDB Buffer Pool Dirty Pages & Page Cleaner Threads

To understand how page flushing stalls occur, visualize the dirty page lifecycle:

Application Writes (Thousands of INSERT / UPDATE per sec)
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│ InnoDB Buffer Pool: Pages marked as "Dirty" in memory       │
└─────────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│ Flush List & LRU (Least Recently Used) Lists                │
└─────────────────────────────────────────────────────────────┘
                         │
         ┌───────────────┴───────────────┐
         ▼                               ▼
[ Default Page Cleaner Setup ]     [ Tuned Multi-Page Cleaners ]
- Only 1 or 4 Page Cleaner Threads - 8 Page Cleaners (1:1 with Pool Instances)
- Low innodb_io_capacity (200 IOPS)- High innodb_io_capacity (15,000 IOPS)
- Dirty pages exceed 75% threshold - Adaptive flushing keeps dirty pages ~20%
- Redo log fills: EMERGENCY HALT!  - ZERO sync stalls! Continuous smooth writes!
- System freezes for 5-10 seconds! - Flat, predictable sub-millisecond latency!

---

Step 1: Diagnosing Buffer Pool Dirty Page Ratio & Flushing Stalls

To determine if your MariaDB server is currently suffering from page cleaner bottlenecks, check dirty page accumulation and log wait metrics:

-- Query Buffer Pool dirty page metrics
SELECT 
    VARIABLE_NAME, 
    VARIABLE_VALUE 
FROM information_schema.GLOBAL_STATUS 
WHERE VARIABLE_NAME IN (
    'Innodb_buffer_pool_pages_dirty',
    'Innodb_buffer_pool_pages_total',
    'Innodb_buffer_pool_wait_free',
    'Innodb_log_waits'
);

Calculate the Dirty Page Percentage: $$\text{Dirty Page %} = \frac{\text{Innodb_buffer_pool_pages_dirty}}{\text{Innodb_buffer_pool_pages_total}} \times 100$$

If Innodb_buffer_pool_wait_free or Innodb_log_waits is greater than 0, your application has already suffered from transaction freezing because no clean buffer pool pages were available!

To inspect page cleaner activity in real time:

SHOW ENGINE INNODB STATUS\G

Locate the BUFFER POOL AND MEMORY section:

----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 34359738368
Dictionary memory allocated 14210401
Buffer pool size   2097152
Free buffers       142
Database pages     1852100
Modified db pages  1489201  <-- OVER 70% OF THE BUFFER POOL IS DIRTY!
Pending reads 0
Pending writes: LRU 0, flush list 412, single page 0
Pages made young 49201, not young 2948102

Notice Pending writes: flush list 412: the page cleaners cannot push dirty pages to disk fast enough!

---

Step 2: System-Wide Tuning in /etc/my.cnf.d/server.cnf for NVMe Storage

On modern dedicated servers equipped with enterprise PCIe NVMe storage, disk write performance can sustain tens of thousands of IOPS without degradation. Legacy defaults designed for spinning disks (like innodb_io_capacity = 200) artificially strangle MariaDB from utilizing the hardware!

Apply the following production parameters in /etc/my.cnf.d/server.cnf:

[mariadb]
# /etc/my.cnf.d/server.cnf
# NextGen Pakistan - High-IOPS InnoDB Page Cleaner & NVMe Flush Tuning Profile

# 1. Align Buffer Pool Instances & Page Cleaners
# For servers with 32GB+ RAM, divide buffer pool into 8 instances to eliminate mutex lock contention.
# Set page cleaners to EXACTLY MATCH the number of buffer pool instances!
innodb_buffer_pool_size = 32G
innodb_buffer_pool_instances = 8
innodb_page_cleaners = 8

# 2. NVMe Drive I/O Capacity Calibration
# Default is 200/2000 (spinning disks). For enterprise PCIe 4.0 NVMe, scale up!
innodb_io_capacity = 10000
innodb_io_capacity_max = 25000

# 3. LRU Scan Depth
# Determines how far down the buffer pool LRU list the page cleaner scans for dirty pages.
# Default is 1024; on fast NVMe storage, set to 2048 to keep a large reserve of clean pages.
innodb_lru_scan_depth = 2048

# 4. Adaptive Flushing and Redo Log Headroom
# Adaptive flushing monitors write velocity and flushes proactively before dirty pages hit 75%.
innodb_adaptive_flushing = ON
innodb_adaptive_flushing_lwm = 10.0   # Low watermark: Start flushing at 10% redo log capacity
innodb_max_dirty_pages_pct = 65.0    # Target maximum percentage of dirty pages
innodb_max_dirty_pages_pct_lwm = 15.0 # Low watermark: Start gentle flushing at 15%

# 5. Flush Method & I/O Threads
# Use direct I/O to bypass Linux OS page cache double buffering
innodb_flush_method = O_DIRECT
innodb_write_io_threads = 8
innodb_read_io_threads = 8

Apply dynamic variables immediately in MariaDB:

SET GLOBAL innodb_io_capacity = 10000;
SET GLOBAL innodb_io_capacity_max = 25000;
SET GLOBAL innodb_lru_scan_depth = 2048;
SET GLOBAL innodb_max_dirty_pages_pct = 65.0;
SET GLOBAL innodb_max_dirty_pages_pct_lwm = 15.0;

---

Step 3: Aligning Redo Log File Size to Absorb Traffic Surges

If your Redo Log capacity is too small, page cleaners are forced into “furious flushing” even if the buffer pool has plenty of room. Ensure your Redo Log can hold at least 1 to 2 hours worth of peak transactional writes:

# Redo Log Configuration
innodb_log_file_size = 4G
innodb_log_files_in_group = 2
innodb_log_buffer_size = 64M

Total redo log capacity will be (4\text{GB} \times 2 = 8\text{GB}). This provides massive headroom for write spikes during high-traffic events without triggering synchronous emergency flushes.

---

Step 4: Monitoring Flush Smoothing & Storage IOPS Depletion

Monitor disk write behavior with iostat to verify that dirty pages are written in a smooth, continuous stream rather than spiky bursts:

iostat -xmt 2

Sample output post-tuning during peak traffic:

Device            r/s     w/s     rkB/s     wkB/s  rrqm/s  wrqm/s  %util
nvme0n1          8.50 1420.00    136.00  84120.00    0.00   45.00  28.40

Notice:

• w/s: 1420.00 writes per second steady and consistent.
• %util: 28.40%: The NVMe drive is handling the write volume comfortably with over 70% headroom remaining.
• Zero query freezing observed in application logs!

---

High-IOPS Dedicated Database Infrastructure in Pakistan

Maintaining consistent, sub-millisecond database query performance under sustained write loads requires dedicated physical storage channels and guaranteed CPU time. Virtualized cloud VPS instances share underlying SSD arrays with dozens of other tenants, causing random write latency spikes that instantly back up MariaDB flush lists.

Deploying on bare-metal Dedicated Servers in Pakistan equips your database cluster with dedicated enterprise NVMe storage arrays in hardware RAID 10, multi-channel ECC DDR5 memory, and dedicated physical CPU cores, guaranteeing that your platforms remain silky-smooth during peak Pakistani commercial surges.