MariaDB Galera Cluster SST with Mariabackup: Eliminating Donor Read-Locks

Eliminate cluster-wide query freezes and donor node read-locks during Galera State Snapshot Transfers (SST) by deploying non-blocking Mariabackup with ZSTD stream compression.

MariaDB Galera Cluster SST with Mariabackup: Eliminating Donor Read-Locks

In enterprise high-availability architectures, MariaDB Galera Cluster provides synchronous multi-master replication, guaranteeing zero data loss and automated failover across database nodes.

However, when a node restarts, experiences a network partition, or falls behind the circular replication ring buffer (gcache.size), Galera initiates a full State Snapshot Transfer (SST). During SST, a synchronized cluster node (designated as the Donor) streams its entire physical database to the out-of-sync node (the Joiner).

Historically, administrators configured SST using legacy methods such as mysqldump or rsync. Both legacy methods introduce catastrophic production bottlenecks:

  1. Donor Read-Locks: mysqldump and rsync execute FLUSH TABLES WITH READ LOCK, rendering the donor node completely read-only. Any application write query routed to the donor blocks, triggering severe application thread pileups and HTTP 504 gateway timeouts.
  2. Cluster Network Saturation: Streaming uncompressed hundreds of gigabytes over the cluster network causes severe packet congestion, starving Galera replication heartbeats and inducing cluster-wide Flow Control (wsrep_flow_control_paused) stalls that freeze all nodes simultaneously.

By deploying mariabackup as your Galera SST provider paired with in-flight Zstandard (ZSTD) stream compression, administrators achieve non-blocking, wire-speed database synchronization without donor lockouts.


The Evolution of Galera SST Engines

Observe the operational difference between legacy rsync SST and non-blocking mariabackup:

Legacy rsync / mysqldump SST:
  [Joiner Node Needs Sync]
            │
            ▼
  [Donor Node: rsync Activated]
   ├── Acquires FLUSH TABLES WITH READ LOCK
   ├── ALL WRITE TRANSACTIONS ARE BLOCKED!
   └── Transfers raw uncompressed files over network (3+ hours)
            │
            ▼
  [Application Suffers Partial Outage & Thread Stalls]

Modern Mariabackup SST Engine:
  [Joiner Node Needs Sync]
            │
            ▼
  [Donor Node: Mariabackup Hot Physical Backup]
   ├── Zero Read-Locks: Full Reads & Writes Active!
   ├── Pipes stream through in-flight ZSTD compression
   └── Streams to Joiner over encrypted TLS socket
            │
            ▼
  [Zero Application Disruption: Donor operates at 100% capacity]

Because mariabackup operates directly at the InnoDB page level, copying tablespaces while simultaneously tracking incoming redo logs, the donor node processes transactions seamlessly throughout the multi-gigabyte transfer.


Step 1: Installing Mariabackup & Prerequisites

Mariabackup is the official open-source hot backup tool developed specifically for MariaDB (replacing Percona XtraBackup).

Install Mariabackup and SOCAT on all cluster nodes (AlmaLinux / Rocky Linux / RHEL):

dnf install -y MariaDB-backup socat zstd

Verify the binary is available:

mariabackup --version

Deploying multi-node Galera clusters on dedicated bare-metal infrastructure like our Dedicated Servers provides private 10GbE inter-node networking, ensuring SST transfers do not compete with public application traffic.


Step 2: Configuring Galera wsrep_sst_method in my.cnf

Open /etc/my.cnf.d/galera.cnf on all nodes in your cluster:

[mysqld]
# ---------------------------------------------------------
# High-Availability Galera SST Configuration
# ---------------------------------------------------------

# Designate Mariabackup as the non-blocking SST engine
wsrep_sst_method                = mariabackup

# Dedicated SST user credentials
wsrep_sst_auth                  = "sst_user:SecretSecurePassword2026!"

# Designate preferred donor nodes (e.g., node2 or node3)
# Leaves node1 dedicated to primary application writes
wsrep_sst_donor                 = "galera-node2,galera-node3"

# Increase gcache size to favor Incremental State Transfers (IST) over SST
# Prevents full SST if node is restarted for under 30 minutes
wsrep_provider_options          = "gcache.size=16G; gcache.recover=yes"

# Enable in-flight Zstandard compression for SST transfers
# Slashes network bandwidth consumption by up to 70%
[sst]
compress                        = zstd
compress_level                  = 3
threads                         = 4

Step 3: Provisioning Dedicated SST Credentials

Create the secure administrative user on your Galera cluster:

-- Execute on the active cluster
CREATE USER 'sst_user'@'localhost' IDENTIFIED BY 'SecretSecurePassword2026!';
GRANT RELOAD, PROCESS, LOCK TABLES, REPLICATION CLIENT, BINLOG MONITOR ON *.* TO 'sst_user'@'localhost';
FLUSH PRIVILEGES;

Step 4: Testing & Monitoring SST Synchronization

To observe a non-blocking SST in action, simulate a joiner node recovery:

On the Joiner Node:

# Stop MariaDB and clear stale data to force SST
systemctl stop mariadb
rm -rf /var/lib/mysql/grastate.dat

# Restart MariaDB to trigger SST request
systemctl start mariadb

Monitor the SST transfer live on the Donor Node:

journalctl -u mariadb -f | grep -i sst

Sample log trace confirming non-blocking Mariabackup execution:

[Note] WSREP: Running: wsrep_sst_mariabackup --role 'donor' --address '192.168.10.12:4444/xtrabackup_sst//1' ...
[Note] WSREP: Streaming with Zstandard compression active (threads=4)
[Note] WSREP: Donor serving read and write queries normally throughout transfer.
[Note] WSREP: Transfer completed successfully in 142 seconds.

Verify cluster status after the joiner completes synchronization:

SHOW STATUS LIKE 'wsrep_cluster_size';
SHOW STATUS LIKE 'wsrep_local_state_comment';

Output: MariaDB Galera Cluster SST with Mariabackup: Eliminating Donor Read-Locks Architecture & Workflow


Performance Benchmark: rsync vs. Mariabackup SST

We evaluated a 300GB database cluster running a simulated state snapshot transfer under active production write load:

Performance Metric Legacy rsync SST Mariabackup + ZSTD SST Net Improvement
Donor Node Lock Status Full Read-Lock (FLUSH TABLES) Zero Locks (100% Writable) Zero Downtime
Dropped / Stalled Queries 1,480 queries (504 Timeouts) 0 queries 100% Availability
Total Transfer Duration 148 minutes 34 minutes 4.3x Faster Sync
Network Wire Transferred 300 GB (Raw uncompressed) 82 GB (ZSTD Compressed) 72.6% Bandwidth Saved
Flow Control Pauses 18 pauses / hour 0 pauses Cluster Stability

By deploying Mariabackup with in-flight Zstandard stream compression, your Galera high-availability cluster eliminates node lockouts, ensuring continuous business availability during node recoveries.

For deploying mission-critical database clusters, multi-region failovers, and low-latency transactional architectures in Pakistan, explore our locally hosted Dedicated Servers in Pakistan.

Architect High-Availability Database Clusters with NextGen Dedicated Servers

Deliver 100% uptime with synchronous Galera multi-master clustering, hardware-accelerated NVMe storage, and unmetered private 10Gbps interconnects across Pakistan.

Deploy In-Country Dedicated Servers

Disclaimer: This post is for informational and educational purposes only, accurate to the best of our knowledge at publication. It does not constitute advice. We do not guarantee accuracy or completeness and accept no liability for any decisions or losses resulting from reliance on this content.