MariaDB Change Data Capture (CDC) with Debezium and Apache Kafka: Real-Time Event-Driven Replication for Enterprise Workloads in Pakistan

Modern enterprise applications across Pakistan—spanning real-time payment processing gateways, fintech ledgers, large-scale supply chain logistics, and high-frequency e-commerce portals—require immediate visibility into database changes. Relying on polling-based architectures (such as querying updated_at > NOW() - INTERVAL 1 MINUTE) introduces devastating database lock contention, wasted I/O bandwidth, and intolerable data propagation delays.

Change Data Capture (CDC) solves this architectural challenge by non-intrusively tailing the database’s native binary commit log (binlog) at line rate. By integrating MariaDB with Debezium and Apache Kafka, engineering teams can capture row-level INSERT, UPDATE, and DELETE mutations with sub-10ms delivery latency, streaming events directly into search indexes (Elasticsearch/OpenSearch), analytical caches (Redis), and data warehouses without impacting transaction throughput.

---

The Architecture of Binlog-Based CDC

Traditional MariaDB replication streams the binary log to replica servers. Debezium operates as a specialized replication client that registers itself as a pseudo-replica with its own unique server-id.

+-------------------------------------------------------------------------+
|                  MariaDB Primary (Dedicated Server)                     |
|                                                                         |
|  [Transactions: ACID] ---> [InnoDB Engine] ---> [Binary Log (ROW)]     |
+-------------------------------------------------------------------------+
                                                       |
                                                       | (Binlog Streaming)
                                                       v
+-------------------------------------------------------------------------+
|                  Debezium MariaDB Connector (Kafka Connect)             |
|                                                                         |
|  [Offset Tracker] ---> [Schema Parser] ---> [JSON / Avro Converter]     |
+-------------------------------------------------------------------------+
                                                       |
                                                       | (Partitioned Events)
                                                       v
+-------------------------------------------------------------------------+
|                    Apache Kafka Distributed Cluster                     |
|                                                                         |
|  Topic: dbserver1.finance.transactions                                 |
|  Topic: dbserver1.ecommerce.orders                                     |
+-------------------------------------------------------------------------+

When operating on mission-critical Dedicated Servers in Pakistan, hosting MariaDB and the Kafka broker on bare-metal hardware with direct-attached NVMe arrays ensures that high-volume CDC event ingestion does not cause binary log serialization stalls or disk queue exhaustion.

---

Step 1: Configuring MariaDB for Row-Based Replication (RBR)

For Debezium to reconstruct the exact “before” and “after” state of mutated rows, MariaDB must be configured with Row-Based Replication (binlog_format=ROW) and full row imaging.

Edit /etc/my.cnf.d/server.cnf under the [mariadb] section:

[mariadb]
# Enable binary logging with unique server ID
server-id = 10101
log_bin = /var/lib/mysql/mariadb-bin
log_bin_index = /var/lib/mysql/mariadb-bin.index
expire_logs_days = 7
max_binlog_size = 1G

# Critical CDC requirements
binlog_format = ROW
binlog_row_image = FULL

# Enable GTID for deterministic failover and offset tracking
gtid_strict_mode = ON

# Ensure transactional integrity
sync_binlog = 1
innodb_flush_log_at_trx_commit = 1

Restart the MariaDB service and verify the settings:

systemctl restart mariadb
mysql -u root -p -e "SHOW VARIABLES LIKE 'binlog_format';"

---

Step 2: Provisioning Dedicated CDC Service Credentials

Create an isolated database user with strict permissions required to read the binlog stream without granting broad administrative privileges:

-- Create Debezium CDC user
CREATE USER 'debezium_cdc'@'10.0.0.%' IDENTIFIED BY 'SuperSecureCDCPassword2026!';

-- Grant required replication and metadata privileges
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT 
ON *.* TO 'debezium_cdc'@'10.0.0.%';

-- Grant lock tables permission for initial schema snapshots
GRANT LOCK TABLES ON *.* TO 'debezium_cdc'@'10.0.0.%';

FLUSH PRIVILEGES;

---

Step 3: Deploying the Debezium Connector on Kafka Connect

Deploy the Debezium connector by submitting a JSON specification to the Kafka Connect REST API (running on port 8083):

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" \
  http://kafka-connect.internal.pk:8083/connectors/ \
  -d '{
    "name": "mariadb-cdc-inventory-connector",
    "config": {
      "connector.class": "io.debezium.connector.mysql.MySqlConnector",
      "tasks.max": "1",
      "database.hostname": "10.0.0.50",
      "database.port": "3306",
      "database.user": "debezium_cdc",
      "database.password": "SuperSecureCDCPassword2026!",
      "database.server.id": "99901",
      "topic.prefix": "pakistan_fintech",
      "database.include.list": "fintech_db",
      "table.include.list": "fintech_db.transactions,fintech_db.settlements",
      "schema.history.internal.kafka.bootstrap.servers": "kafka1.internal.pk:9092,kafka2.internal.pk:9092",
      "schema.history.internal.kafka.topic": "schema-changes.fintech_db",
      "database.ssl.mode": "required",
      "snapshot.mode": "initial",
      "tombstones.on.delete": "true"
    }
  }'

---

Step 4: Consuming Real-Time CDC Mutation Events

Once the connector transitions to RUNNING, Debezium publishes events to Kafka topics following the convention: <topic.prefix>.<database>.<table>. Each message payload contains the comprehensive transaction metadata:

{
  "before": {
    "transaction_id": 482910,
    "user_id": 1402,
    "amount": 2500.00,
    "status": "PENDING"
  },
  "after": {
    "transaction_id": 482910,
    "user_id": 1402,
    "amount": 2500.00,
    "status": "SETTLED"
  },
  "source": {
    "version": "2.7.0.Final",
    "connector": "mysql",
    "name": "pakistan_fintech",
    "ts_ms": 1790931200150,
    "db": "fintech_db",
    "table": "transactions",
    "server_id": 10101,
    "gtid": "0-10101-948210"
  },
  "op": "u",
  "ts_ms": 1790931200165
}

The op field indicates the operation (c for create, u for update, d for delete), giving downstream microservices deterministic, change-aware event triggers.

---

Diagnosing Binlog Lag and Connector Health

To verify connector throughput and ensure zero binlog replication lag:

# Check Kafka Connect task status
curl -s http://kafka-connect.internal.pk:8083/connectors/mariadb-cdc-inventory-connector/status | jq .

# Monitor MariaDB binlog disk consumption
mysql -u root -p -e "SHOW BINARY LOGS;"

Deploying real-time event streaming architectures on bare-metal Dedicated Servers provides dedicated NVMe write throughput, multi-gigabit private interconnects, and unmetered network bandwidth to prevent I/O serialization bottlenecks across enterprise MariaDB clusters.