Spool and Waiting to Deliver

The email spool is a list of emails, in order of when they are created, that are available for the server to send out to other mail servers or to deliver locally. SmarterMail is multithreaded, which means that if a message cannot process out of the spool, SmarterMail simply moves on to the next message until the maximum number of threads that are designated in the administrative configurations are in use. Administrators can use the information here to adjust threads and resources to allocate for concurrent messages.

Messages enter and leave the spool fairly quickly. In fact, some pass through so quickly that they will not display in the spool. Most messages in the spool are displayed because they are large, have many recipients, or are having trouble being sent to their final destination.

The Spool tab displays all inbound and outbound messages, including ones that are attempting to be delivered or waiting to be delivered. To view a filtered display of the spool for only messages that are waiting to be delivered, use the Waiting to Deliver tab. Both tabs, along with the rest of the Spool Management area, are exposed only to System Administrators – domain administrators do not have access to the spool, since it spans every domain hosted on the server.

Important Notes:

Messages in “Waiting to Deliver” status usually indicate that one or more recipients encountered a delivery error, and the system is holding them until the next retry interval. Emails stuck in local delivery or waiting without retry attempts are often caused by an I/O bottleneck at the CPU or storage array.
The Spool and Waiting to Deliver tabs can display a maximum of 50,000 messages combined by default. For example, if 20,000 messages appear in Spool and 30,000 in Waiting to Deliver, the total display limit has been reached. When the combined count hits this limit, there are likely more messages in each category than can be shown. This limit is tracked separately from the Spam Quarantine and Virus Quarantine tabs, which default to a combined cap of 5,000 items.
The tab counts shown at the top of the page reflect every item SmarterMail has detected, but the grid itself only loads a subset of rows at a time (sorted and paged on demand) for the sake of server performance. For example, if a tab shows 5,000 items, the grid will still only render the current page of results as you scroll or page through it – it isn't rendering all 5,000 rows in the browser at once. To review every item at the file-system level rather than through the grid, open the corresponding spool, Spam Quarantine, or Virus Quarantine directory directly on the server.

The following details can be seen for each entry in the spool:

Filename - The unique name of the EML file on the hard disk of the SmarterMail server.
Spool Path - The folder the message currently resides in. If Subspools (found in Settings > General) is set to a value of 2 or higher, SmarterMail splits the spool directory into that many SubSpool0, SubSpool1, etc. subfolders and rotates new messages across them round-robin as they arrive, instead of writing every message into a single spool folder. This is purely a performance and file-system optimization for very high-volume servers – a directory with tens of thousands of files in it can slow down disk I/O, so spreading messages across several subfolders keeps each individual folder smaller. The Spool Path column tells you which physical subfolder (or the root spool folder, if subspools are disabled) a given message's EML and HDR files are sitting in, which is useful if you ever need to locate the file directly on disk.
Sender - The email address that initially sent the email.
Recipients - The number of delivered/total recipients.
Size - The total size of the message on the hard drive, in kilobytes.
Attempts - The number of delivery attempts that have been made.
Time in Spool - The total amount of time the message has been in the spool.
Priority - The priority level of the message: low, normal or high.
Status - The current status of the message. As a message moves through the spool it can pass through several distinct statuses:
- Receiving - A very brief, initial status that only appears while SmarterMail is actively writing the incoming message to disk. Messages rarely stay in this status long enough to be seen in the grid.
- Delivery Delay - This is effectively the first status of any message that lingers in the spool. Administrators can configure a Delivery Delay (in seconds) within the system's General Settings – internally this value is called the relay interval. This delay represents the amount of time mail will be held in the spool before delivery processing begins. A delivery delay is beneficial when you are running a secondary service (such as a virus checker) that needs access to messages prior to delivery, as it provides ample time for the secondary service to interact with the message.
- Spam Check - At this stage, SmarterMail runs the configured spam checks against the contents of the email. Messages from whitelisted senders bypass this delivery status entirely.
- Command-Line - An optional stage that only appears if a command-line antivirus or scanning executable has been configured under Antivirus settings. SmarterMail hands the message off to that external process and waits for it to finish before continuing.
- Local Delivery - The message is currently being written into a local user's mailbox on this server (i.e., the sender and recipient share the same SmarterMail installation).
- Waiting to Deliver / Delivered / Throttled - Once a message reaches the remote-delivery stage, its displayed status depends on how many recipients are still pending. If every recipient has been handled, it shows as Delivered (and drops out of the spool). If one or more recipients are still pending because a prior delivery attempt failed, it shows as Waiting to Deliver and sits idle until its next retry interval arrives. If the sending user, mailing list, or domain has hit a configured sending limit, it instead shows as Throttled until the throttle window clears.
- Remote Delivery - The message is actively being transmitted to a remote mail server outside of this SmarterMail installation (an active SMTP session is in progress).
Note: Quarantined messages (held by spam or virus scanning) do not appear in the Spool or Waiting to Deliver grids at all – they are broken out into their own Spam Quarantine and Virus Quarantine tabs.
Next Attempt - The date and time of the next delivery attempt. By default, SmarterMail retries a failed remote delivery four times, waiting 15 minutes, 30 minutes, 60 minutes, and 90 minutes after each successive failure. If the final retry also fails, the message is bounced back to the sender as undeliverable rather than retried again.

To view the contents of a message or its intended recipients, click on the entry's row. The email will load in a popup window. If you are presented with a note that the "Message no longer exists," it's possible that the message was already delivered or removed by antivirus software or that the spool contains an orphaned HDR file without the associated EML.

The following actions can be taken on selected entries using the Actions (⋮) button:

Force - Pushes the selected message(s) to the front of the delivery queue. Force does more than just raise priority: it sets the message's priority to High, bypasses any remaining Delivery Delay, bypasses throttling for that message, and moves its next-attempt time into the past so it becomes immediately eligible for delivery on the next pass through the spool. In effect, Force skips the message straight to the front of the line and clears the obstacles that were holding it back.

Note: The status of forced messages will not update in the grid until the server actually passes through the spool again, so don't be alarmed if the row doesn't change immediately after clicking Force.

Example: a VIP user reports that an urgent, time-sensitive message has been sitting in Waiting to Deliver because the recipient's mail server was briefly unreachable. Rather than waiting for the next scheduled retry interval, select the message and click Force to push it out immediately once the remote server is confirmed to be back online.

Reset Retries - Resets the retry count and next-attempt time on the selected message(s), effectively starting the delivery process over from the beginning (including its Delivery Delay). Internally, this clears the message's attempt counter back to zero and sets its next-attempt time to the current time, so the message becomes eligible for another delivery attempt as soon as the spool cycles through it – it does not, by itself, force the message to the front of the queue the way Force does. This is useful if a DNS or firewall problem has been recently resolved, or if you are using SmartHosting and the target server was down; without a reset, a message that already exhausted all four retry attempts (15, 30, 60, and 90 minutes) would otherwise be bounced back to the sender instead of being tried again.
Change Priority - Changes the priority level of the selected message(s) to Low, Normal, or High. Higher priority messages are processed ahead of lower priority ones as spool threads become available, but priority alone does not bypass a message's Delivery Delay or retry timing the way Force does.
Move Messages - Moves the location of the selected message(s) from the spool directory to a new path on the server. The dialog defaults to a MovedSpoolItems subfolder under the configured spool path, or any other folder path on the server can be entered. Moving the .eml files out of the active spool is useful because it allows an administrator to review the messages, forward them, or hand them off for analysis without the delivery engine continuing to retry them in the background.

Example: if a domain is suspected of being compromised and blasting spam through the spool, moving those messages out of the active spool immediately stops SmarterMail from continuing to attempt delivery on them, buying time to investigate and lock down the account before deciding whether to delete or release them.

Download EML - Downloads the raw EML file for the message (or a .zip archive if multiple messages are selected), letting the message be opened, forwarded, or inspected in an external tool.
Delete - Removes the selected message(s) from the spool.

Note: A confirmation dialog will display before the delete is carried out, listing how many messages will be removed. Deletion is still permanent and cannot be undone once confirmed, so review the selection carefully before clicking through the prompt.

Delete All from Sender - Deletes all messages in the spool from the specified sender. Caution should be used with this action as the same email address can be used for different types of messages. For example, a "no-reply" address can be used for bulk notifications but may also be used for promotional emails, etc. Deleting all messages from the no-reply sender may, therefore, have unintended consequences.
Move All from Sender - Moves all messages in the spool from the specified sender. The same caution should be used as with deleting all messages from sender.

Searching the Spool

System administrators can search for messages from particular senders in the spool. To do so, use the Search bar at the top of the content pane. Simply type in the email address of the sender and click the magnifying glass to search for any messages from that sender that are in the spool.