Few error messages in Linux mail administration create more confusion than Exim’s notorious delivery deferral:
retry time not reached for any host after a previous delivery failureWhen this status appears in /var/log/exim_mainlog, administrators often test the destination mail server manually, confirm that the network path is open, and yet watch in frustration as Exim refuses to attempt outbound SMTP delivery. Outgoing messages pile up in the spool directory, users report delayed transactional emails, and manual queue runs appear to do nothing.
This behavior is not an Exim malfunction; it is the calculated result of Exim’s internal retry database architecture and exponential backoff engine.
This runbook provides an exhaustive, root-cause diagnostic guide for systems engineers, cPanel/WHM administrators, and DevOps teams. We explore the internal state machine of Exim’s hints databases, dissect the five core root causes triggering delivery deferral, provide hands-on CLI diagnostic workflows, and execute clean remediation without corrupting the mail spool.
1. Under the Hood: Exim’s Retry Database & Routing Architecture
To resolve this issue permanently, you must understand how Exim decides whether to open a TCP socket or instantly defer a message.
+-------------------------------------------------------------------+
| New Outbound Email |
+-------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------+
| Remote MX Host Resolution (DNS) |
+-------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------+
| Query Exim Retry Database (/var/spool/exim/db/retry) |
+-------------------------------------------------------------------+
/ \
[ Record Found & [ No Record OR
Now < Next Retry Time ] Now >= Next Retry Time ]
/ \
v v
+--------------------------------+ +-------------------------------+
| Instant Deferral: | | Attempt Outbound SMTP |
| "retry time not reached" | | TCP Handshake (Port 25) |
| (No TCP Socket Opened) | +-------------------------------+
+--------------------------------+ |
v
+-------------------------------+
| Connection Succeeded or Failed|
+-------------------------------+
|
[ If Failed (Timeout/4xx) ]
|
v
+-------------------------------+
| Write Failure & Update Next |
| Retry Timestamp in retry DB |
+-------------------------------+The Exim Hints Database (/var/spool/exim/db/retry)
Exim maintains persistent delivery telemetry inside a collection of Berkeley DB files located in /var/spool/exim/db/ (on cPanel/RHEL/AlmaLinux) or /var/spool/exim4/db/ (on Debian/Ubuntu). The primary file governing delivery schedules is:
/var/spool/exim/db/retry(Berkeley DB format)/var/spool/exim/db/retry.lockfile
Whenever Exim fails to deliver an email to a remote host—due to a TCP connection timeout, a temporary SMTP 4xx response code, or a DNS resolution failure—it does not simply queue the message for immediate re-attempt. Instead, it creates an entry in the retry hints database.
The record stores:
- Target Key: The destination domain or specific remote MX host IP (e.g.,
T:aspmx.l.google.com:142.250.153.26). - First Failure Timestamp: When the host first began failing.
- Last Try Timestamp: The most recent delivery attempt timestamp.
- Next Try Timestamp: The precise future time before which Exim is strictly forbidden from opening a new network connection to this host.
- Error Code & Reason: The recorded underlying failure (e.g.,
Connection refused,Connection timed out, or451 Temporary local problem).
The Exponential Backoff Formula
Exim calculates the Next Try Timestamp based on retry rules defined in /etc/exim.conf (or WHM’s Exim Configuration Manager). A standard default rule resembles:
* * F,2h,15m; G,16h,1h,1.5; F,4d,6hThis configuration breaks down into three distinct phases:
- First 2 hours (
F,2h,15m): Fixed interval retry every 15 minutes. - Next 16 hours (
G,16h,1h,1.5): Geometric exponential backoff starting at 1 hour, multiplying the interval by 1.5 after each failure. - Up to 4 days (
F,4d,6h): Fixed interval retry every 6 hours until the message reaches maximum age and bounces.
The Catch-22: Why Network Fixes Don’t Instantly Flush the Queue
When a temporary network glitch, firewall block, or DNS outage occurs, Exim marks the target MX host as failing. If the failure repeats across several attempts, the next retry window may be pushed 1 hour, 3 hours, or 6 hours into the future.
If you fix the underlying network issue five minutes later, Exim will still refuse to send mail to that host.
During standard queue runners (exim -q), Exim inspects the retry database before creating a network socket. Because Current Time < Next Try Time, Exim immediately skips the host and logs:
H=aspmx.l.google.com [142.250.153.26]: retry time not reached for any host after a previous delivery failureNo TCP packets leave your server. The message remains deferred until the retry database is either cleared or the scheduled timer expires.
2. The 5 Core Root Causes of Exim Delivery Deferral
While the retry database enforces the delay, it is never the initial cause of the problem. A prior failure must occur to populate the database. Below are the five primary architectural root causes.
| Root Cause | Network Layer | Typical Log Clue | Diagnostic Command |
|---|---|---|---|
| 1. Cached Transient Outage | Application State | Initial failure logged hours ago; target is now live | /usr/sbin/exim_dumpdb /var/spool/exim retry |
| 2. Outbound Port 25 Filtering | Network / Firewall | Connection timed out across all destination MXs |
nc -zv aspmx.l.google.com 25 |
| 3. rDNS / PTR Mismatch | DNS / IP Reputation | Remote MTA rejects with 421, 450, or 550 codes | dig -x <outgoing-ip> +short |
| 4. DNS Resolver Latency / DNSBL | Transport / DNS | lookup deferred: DNS query timed out |
time dig mx <remote-domain.com> |
5. mailips / mailhelo Mismatch |
Application / Transport | Hostname in EHLO differs from PTR / outbound IP | Inspect /etc/mailips & /etc/mailhelo |
Cause 1: Cached Temporary Connection Failures
The initial failure was a momentary routing flutter, upstream packet drop, or a short restart of the recipient’s mail exchanger. Even though the remote server resumed normal operations within 60 seconds, Exim logged the failure and set a multi-hour backoff window.
Cause 2: Outbound Port 25 Filtering & Firewall Restrictions
SMTP communication between Mail Transfer Agents requires outbound TCP port 25. Many residential ISPs, cloud hyperscalers, and hosting providers block outbound port 25 by default to curb spam.
On cPanel servers, local firewall tools such as ConfigServer Security & Firewall (CSF) include protection rules:
SMTP_BLOCK = "1": Restricts outbound SMTP solely to local daemons (mailman, exim, root).SMTP_ALLOWLOCAL = "1": If misconfigured or if iptables rules get corrupted, legitimate Exim child processes are prevented from making outbound SYN requests to external port 25.
Cause 3: Reverse DNS (PTR) Record Mismatches
Major email service providers (Google, Microsoft 365, Yahoo, ProtonMail) enforce strict sender verification standards. If your mail server initiates an SMTP connection from secondary IP 198.51.100.25, the receiving MTA queries the PTR record for 198.51.100.25.
- If the PTR record is missing, points to a generic ISP hostname (e.g.,
25-100-51-198.static.provider.net), or does not resolve back to the exact same IP (Forward-Confirmed Reverse DNS), remote servers will return temporary deferral codes (e.g.,421 4.7.0 [IP] Our system has detected suspicious traffic) or reject the connection outright. - Exim logs this 4xx temporary rejection, writes it to the retry database, and marks the host as deferred.
Cause 4: Upstream DNS Resolver Latency & DNSBL Timeouts
Exim performs multiple DNS lookups per delivery: MX records, A/AAAA records for each MX, and SPF/DKIM verification lookups.
If /etc/resolv.conf points to sluggish public resolvers that rate-limit your queries or suffer packet loss, Exim’s internal resolver timer trips before the socket can be initiated. A DNS resolution timeout is classified as a temporary routing failure, which is immediately recorded in /var/spool/exim/db/retry.
Cause 5: Exim mailips and mailhelo Configuration Mismatches
On multi-IP cPanel servers, administrators frequently bind outgoing mail to specific account IPs using /etc/mailips and /etc/mailhelo.
If an account is assigned outbound IP 198.51.100.50, but /etc/mailhelo instructs Exim to present mail.primarydomain.com (which resolves to 198.51.100.10), destination MTAs detect an EHLO/rDNS identity clash. Sensitive receiving filters defer the handshake with a 451 or 421 response, triggering Exim’s retry backoff.
3. Step-by-Step Diagnostic Procedures
When diagnosing a deferred mail queue, follow this disciplined diagnostic sequence to pinpoint whether the root cause is cached data, firewall blocks, DNS failures, or remote policy rejections.
Step 3.1: Inspect the Mail Log with exigrep
Locate a deferred message in the queue using exim -bp and isolate its delivery history using exigrep:
# List deferred messages in the queue
exim -bp | grep -B1 "retry time not reached"
# Extract the full transaction history for a specific Message-ID
exigrep 1tABCD-000123-XY /var/log/exim_mainlogLook for the first attempt to the destination host. You will typically find the underlying trigger that preceded the deferrals:
2026-10-15 08:12:04 1tABCD-000123-XY H=aspmx.l.google.com [142.250.153.26]: Connection timed out
2026-10-15 08:12:04 1tABCD-000123-XY == [email protected] R=lookuphost T=remote_smtp defer (110): Connection timed out
2026-10-15 08:27:12 1tABCD-000123-XY == [email protected] R=lookuphost T=remote_smtp defer (-53): retry time not reached for any host after a previous delivery failureIn this output, defer (110): Connection timed out reveals that a network or port block caused the initial failure, while defer (-53) is the resulting retry suppression.
Step 3.2: Inspect the Exim Retry Database (exim_dumpdb)
Use Exim’s built-in database utility to view the exact contents of the retry cache:
/usr/sbin/exim_dumpdb /var/spool/exim retryThe output displays entries formatted in columns:
T:aspmx.l.google.com:142.250.153.26 110 Connection timed out
15-Oct-2026 08:12:04 15-Oct-2026 08:27:12 15-Oct-2026 09:12:04 *Columns represent:
- Key: Transport/Host/IP combination.
- Error Code & Text:
110 Connection timed out. - First Failed: Timestamp of the initial failure.
- Last Tried: Timestamp of the most recent probe.
- Next Try: Timestamp before which Exim will suppress all delivery attempts (
15-Oct-2026 09:12:04). - Expired Flag: An asterisk (
*) indicates that the record is active and actively deferring traffic.
Step 3.3: Test Outbound Port 25 Connectivity
To determine whether outbound port 25 is open or blocked by your hosting provider, test direct TCP connectivity to a known reliable MX host:
# Test raw TCP handshake
nc -zv aspmx.l.google.com 25Expected Successful Output:
Connection to aspmx.l.google.com 25 port [tcp/smtp] succeeded!Failure Output (Port 25 Blocked or Filtered):
nc: connect to aspmx.l.google.com port 25 (tcp) timed out: Operation now in progressIf the connection succeeds, test the TLS handshake using openssl:
openssl s_client -starttls smtp -connect aspmx.l.google.com:25 -crlfIf openssl completes the handshake and displays the remote server’s 220 banner, outbound network transit is intact. The deferral is therefore caused by stale cache records or IP-reputation/PTR policies.
Step 3.4: Verify Reverse DNS (PTR) Alignment
Identify the outgoing IP configured for your domain and verify its PTR record:
# Find which IP Exim uses for the domain (cPanel)
grep "yourdomain.com" /etc/mailips
# Query PTR record for this IP
dig -x 198.51.100.25 +short
# Query forward A record for the resulting hostname
dig A mail.yourdomain.com +shortBoth lookups must match symmetrically:
198.51.100.25-> resolves tomail.yourdomain.commail.yourdomain.com-> resolves to198.51.100.25
If the PTR resolves to an incorrect hostname or fails completely, remote servers will continue to reject deliveries, repopulating the retry database after every flush.
4. Actionable Remediation Runbook
Once the underlying issue (firewall, DNS, or PTR) has been addressed, follow this runbook to purge the retry database and process queued emails.
4.1: Purge Stale Exim Retry Database Entries
To flush the retry hints database without terminating active mail services, use exim_tidydb. Setting the threshold age to zero days (0d) removes all deferred host entries:
# Remove all records older than 0 days from the retry DB
/usr/sbin/exim_tidydb -t 0d /var/spool/exim retryEmergency Hard Reset (Corrupted Berkeley DB)
If exim_tidydb reports database locking errors or File exists corruption, perform an atomic cleanup of the hints database:
# 1. Stop Exim to release file locks
systemctl stop exim
# 2. Remove the retry Berkeley DB and lockfiles
rm -f /var/spool/exim/db/retry*
rm -f /var/spool/exim/db/wait-*
# 3. Restart Exim (Exim will automatically regenerate clean DB files)
systemctl start exim4.2: Force Queue Processing & Debug Delivery
With the retry database cleared, you can force Exim to re-evaluate the mail queue immediately.
Test a Single Message in Verbose Mode
Run a verbose test delivery on a single message ID to observe the real-time SMTP transaction and verify that the fix works:
exim -v -M 1tABCD-000123-XYHealthy SMTP Transaction Trace:
Connecting to aspmx.l.google.com [142.250.153.26]:25 ... connected
SMTP<< 220 mx.google.com ESMTP ...
SMTP>> EHLO mail.yourdomain.com
SMTP<< 250-mx.google.com at your service
SMTP>> MAIL FROM:<[email protected]> SIZE=2048
SMTP<< 250 2.1.0 OK
SMTP>> RCPT TO:<[email protected]>
SMTP<< 250 2.1.5 OK
SMTP>> DATA
SMTP<< 354 Go ahead
SMTP>> .
SMTP<< 250 2.0.0 OK 1729000000 - gsmtp
SMTP>> QUIT
LOG: MAIN
=> [email protected] R=lookuphost T=remote_smtp H=aspmx.l.google.com [142.250.153.26] C="250 2.0.0 OK"Force All Queued and Frozen Messages
To ignore all remaining retry timers and attempt delivery on every message in the spool:
# -q: queue runner, -f: force retry, -f (second): also retry frozen messages
exim -qff -vDelete Spam or Backscatter Floods
If your queue contains thousands of unauthorized or spoofed spam messages generated prior to securing your server, purge them to prevent server exhaustion:
# Remove a specific message from queue
exim -Mrm 1tABCD-000123-XY
# Remove all frozen messages from the queue
exipick -z -i | xargs exim -Mrm
# Remove all messages from a compromised sender email
exipick -f "[email protected]" -i | xargs exim -Mrm4.3: Tune Exim Retry Rules in WHM / exim.conf
If your server delivers to remote services that frequently enforce rate limits (such as corporate Exchange servers or regional ISPs), default retry intervals may be overly aggressive.
In WHM > Service Configuration > Exim Configuration Manager > Advanced Editor, locate the begin retry section and adjust your host rules:
begin retry
# Rule syntax:
# <target> <error> <rule1> ; <rule2> ; ...
# Aggressive retry for local intra-server routing
*@localhost * F,1h,5m
# Gentle retry backoff for remote destinations
* * F,1h,10m; G,8h,30m,1.5; F,3d,4hThis configuration checks failed remote hosts every 10 minutes during the first hour, expands the interval up to 8 hours with a 1.5x backoff, and limits final attempts to every 4 hours for up to 3 days.
4.4: Configure an Outbound Smarthost (When Port 25 is Blocked)
If your upstream network provider or datacenter restricts outbound TCP port 25 and will not remove the block, configure Exim to route outbound emails through an authenticated smart host (such as SendGrid, Amazon SES, or Mailgun) over alternate ports (587 or 465).
In cPanel WHM (Exim Configuration Manager > Basic Editor > Mail):
- Smarthost route: Enter
* smtp.sendgrid.net::587. - AUTH section (
begin authenticators):
sendgrid_login:
driver = plaintext
public_name = LOGIN
client_send = : apikey : YOUR_SECRET_API_KEY- Save and restart Exim. This routes all outbound mail through port 587, bypassing local port 25 blocks entirely.
5. Architectural Infrastructure: The Multi-Tenant Shared IP Dilemma
System administrators frequently encounter a recurring cycle: Exim retry databases are flushed, queues clear, and yet within 48 to 72 hours, the deferral warnings return.
In 90% of recurring cases on shared cPanel servers, this is caused by IP reputation contamination.
+-------------------------------------------------------------------------+
| Shared Multi-Tenant Server (Single IP) |
| |
| [ Account A: High-Value ] [ Account B: Compromised WordPress ] |
| Corporate ERP Invoices Sending 50,000 Phishing Emails |
+-------------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------------+
| Shared Outgoing IPv4: 198.51.100.50 |
+-------------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------------+
| Remote MTAs (Gmail, Outlook, Yahoo) Blacklist IP 198.51.100.50 |
+-------------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------------+
| All Corporate Invoices from Account A Are Deferred (421 / 450) |
| Exim Retry DB Locks: "retry time not reached for any host" |
+-------------------------------------------------------------------------+The Cost of Shared IP Pools
When hundreds of websites share a single outbound IP address:
- Noisy Neighbor Penalty: A single compromised WordPress plugin on an unrelated cPanel account can transmit thousands of spam messages within minutes.
- Automated Blacklisting: Major DNSBLs (Spamhaus, Barracuda, Invaluement) and provider reputation engines flag the shared IP.
- Queue Choke: Once remote MTAs begin throttling or issuing 4xx temporary rejections to the shared IP, Exim’s retry database locks out all subsequent outbound emails across every domain on that server.
The Dedicated Bare-Metal Solution
Mission-critical business emails, financial transactional receipts, and ERP notifications require strict hardware isolation, dedicated IP ranges, and full infrastructure control.
Migrating your core mail handling and hosting workloads to enterprise bare metal—such as high-performance international Dedicated Servers or locally routed Dedicated Servers in Pakistan—eliminates the shared IP vulnerability:
- Isolated IP Allocations: Clean, dedicated IPv4 subnets assigned exclusively to your organization, protecting your reputation from noisy neighbors.
- Direct rDNS / PTR Control: Automated forward and reverse DNS configuration via datacenter control panels ensures full alignment with SPF, DKIM, and DMARC policies.
- Unrestricted Network Transit: Dedicated 1Gbps / 10Gbps uplinks with unfiltered outbound port 25 transit directly peered to major tier-1 backbones and local internet exchanges (PkIX).
- Dedicated Compute & Disk I/O: High-speed NVMe storage arrays process thousands of concurrent Exim queue read/writes without locking the Berkeley DB hints engine.
6. Comprehensive Sysadmin Diagnostic Checklist
Keep this checklist handy whenever troubleshooting an Exim deferral crisis:
| Phase | Action | Shell Command |
|---|---|---|
| 1. Identify | Locate deferred messages | exim -bp | grep "retry time not reached" |
| 2. Trace | Extract full history of failed message | exigrep <message-id> /var/log/exim_mainlog |
| 3. Inspect | Read the active retry database records | /usr/sbin/exim_dumpdb /var/spool/exim retry |
| 4. Test Network | Verify outbound port 25 connectivity | nc -zv aspmx.l.google.com 25 |
| 5. Test TLS | Verify STARTTLS SMTP handshake | openssl s_client -starttls smtp -connect aspmx.l.google.com:25 |
| 6. Check PTR | Confirm reverse DNS resolution matches A record | dig -x <outgoing-ip> +short |
| 7. Flush Cache | Purge stale retry entries cleanly | /usr/sbin/exim_tidydb -t 0d /var/spool/exim retry |
| 8. Force Run | Re-attempt delivery with verbose logging | exim -v -M <message-id> |
| 9. Process Queue | Attempt delivery on all deferred items | exim -qff |
Eliminate Mail Deferrals with Dedicated Bare-Metal Servers
Tired of shared IP blacklists and stuck Exim queues? Nextgen provides mission-critical enterprise infrastructure with dedicated clean IP allocations, automated rDNS/PTR management, and low-latency local routing.
