Troubleshooting

** These settings are managed in the hub in SmarterMail HA. All settings are available when running SmarterMail Enterprise.

SmarterMail makes managing the mail server a breeze by isolating the monitoring and management aspects from the setup and configuration. In the Troubleshooting section, administrators can access settings, tools and dashboards that will help them better understand what's occurring on their mail server and quickly take action while troubleshooting any issues that may arise.

A major part of troubleshooting issues is logging. By default, SmarterMail logs virtually every process and protocol available within the system. Having these logs means that, when issues DO occur, administrators can quickly and easily find out the what's going on and get the problems resolved. If nothing else, having access to logs makes working with SmarterTools much easier as it gives our support agents access to information that can then be used to further find and fix issues, or work with our developers to figure out what's going on so a fix can be implemented.

That said, logs CAN take up space on a server. By default, most of SmarterMail's log levels are set to "Exceptions Only". This means that the logs will capture and write out errors but not details. This keeps the log files small. At the other end of the spectrum, Detailed keeps the most amount of information available, but also means the log files can get quite large, quite quickly. However, this gives administrators the most information possible to help find the root cause of a problem.

Note: Not every log is Exceptions Only out of the box. A handful of high-value logs — SMTP, Delivery, Maintenance, Error, and Licensing among them — ship set to their most verbose logging tier by default so that support staff and administrators always have a detailed trail for the two things that generate the most support tickets: mail flow and unexpected errors. Keep this in mind when estimating disk usage, since SMTP and Delivery logs are typically the largest and busiest logs on the server even without an administrator changing anything.

System administrators have access to the following Troubleshooting tabs:

Options - Configure the log and indexing settings for the server
View Logs - Review the logs to look for errors or monitor recent activity
Services - Enable or disable specific services, including IMAP, SMTP, etc.
Mailbox Indexing - View the status of user indexing occurring on the server
Mailbox Migration - View the mailbox migrations occurring on the server

Options

Use this section to manage how the logs are written and to customize the indexing configuration:

Log Files

Compress Log Files After** - The number of days a log file will sit uncompressed before SmarterMail automatically zips it to save disk space. Compressed logs remain searchable and downloadable from the View Logs tab; they are simply stored more efficiently. By default, this is enabled and set to 3 days, meaning a log file that hasn't been touched in three days will be compressed automatically. For example, on a busy server the current day's SMTP log might be several hundred megabytes uncompressed, but once it ages past the three-day threshold it will typically shrink to a fraction of that size once zipped.
Delete Log Files After** - The number of days after which log files are automatically and permanently deleted. By default, this is enabled and set to 60 days. Once a log file is deleted this way it cannot be recovered, so administrators who need to retain logs for compliance or auditing purposes longer than 60 days should either increase this value or export/download the relevant logs before they age out.
Debug Log IDs (one per line) - This section should only be used when instructed by SmarterTools Support. In order to better troubleshoot an issue within SmarterMail, SmarterTools Support may require additional logging. In this section, Debug Log IDs can be entered. Entering a log ID in this box will enable the creation of a separate log file which will contain information Support needs for troubleshooting. To disable a specific log, simply remove it from the list.

Note: The Compress and Delete settings above work together but independently — a log can be compressed at 3 days and still sit on disk, untouched, until it reaches the 60-day deletion threshold. If disk space is a concern (for example, on a VPS with a small system drive), consider lowering the Delete Log Files After value rather than disabling compression, since compression alone will not reclaim as much space as removing old logs entirely.

Protocol Logging

Use this section to adjust the log detail level for each protocol SmarterMail supports. When set to Exception Only, SmarterMail will produce small-sized logs that record only errors. When set to Normal, SmarterMail will produce medium-sized logs that record most activity taken on the mail server. When set to Detailed, SmarterMail will produce log files that can get very large and contain extensive logging. Only change logs to Detailed when asked by SmarterTools Support or when actively troubleshooting a specific issue — then change it back to Exception Only or Normal once you've captured what you need.

Most protocol logs default to Exception Only, but SMTP and XMPP are the exceptions to that rule — both ship set to the deepest logging tier out of the box, since mail delivery problems and chat connectivity issues are common troubleshooting targets. Also note that if you set EAS, EWS, or MAPI logging more verbose than Normal, SmarterMail will display an on-screen warning reminding you that detailed logging for these protocols can generate a large volume of data very quickly (client sync protocols like ActiveSync and MAPI tend to poll or push data far more frequently than SMTP or POP sessions do).

The following log file types can be adjusted:

Autodiscover - The log level for Autodiscover. Useful for helping figure out why a particular user can't automatically connect to an email client.
EAS - The log level for EAS connections. Useful for helping find issues with things like why a user on an iPhone is having an issue syncing their calendar properly, etc.
EWS - The log level for EWS sessions. Useful for helping find issues trying to connect to a client such as Apple Mail.
IMAP - The log level for IMAP sessions. Useful for helping to figure out why a user can't connect to any email client that supports IMAP. (IMAP Proxy logging is managed on the hub for SmarterMail HA.)
LDAP - The log level for LDAP sessions. Useful for helping find issues when using Active Directory as an authentication method. (IMAP Proxy logging is managed on the hub for SmarterMail HA.)
MAPI - The log level for MAPI sessions. Useful for helping find issues trying to connect a user to a client such as Microsoft Outlook 2019 for Windows.
OAB - The log level for Offline Address Book (OAB). Useful for helping find issues with a user's offline address book, especially when using Microsoft Outlook.
POP - The log level for POP sessions. Useful for helping to figure out why a user can't connect to any email client that supports IMAP. (IMAP Proxy logging is managed on the hub for SmarterMail HA.)
Sharepoint - The log level for Sharepoint Sync (Add to Outlook). Useful for helping to figure out why a client can't connect to any email client that supports Sharepoint Sync.
SMTP - The log level for SMTP sessions. Useful for helping figure out why a message wasn't delivered to a recipient, and helps ensure the message was, in fact, sent by the user. (IMAP Proxy logging is managed on the hub for SmarterMail HA.)
WebDAV - The log level for CalDav and CardDav sessions. Useful for helping to figure out why a calendar or contacts app can't connect to any email client that supports CalDAV or CardDAV.
XMPP - The log level for Live Chat and online meetings. Useful for helping with issues such as a user that is unable to connect to a live chat client. (IMAP Proxy logging is managed on the hub for SmarterMail HA.)

Note: More detailed logs require more disk space. If you choose a detailed log, you may want to enable the auto-delete setting on the Options tab.

Process Logging

Use this section to adjust the log detail level for internal processes within SmarterMail (as opposed to the client-facing protocols covered above). The same three-tier scale applies: Exception Only produces the smallest logs and records only errors; Normal records most activity; Detailed produces the largest, most complete logs and should only be used when asked by SmarterTools Support or when actively troubleshooting server operations.

As with Protocol Logging, not every process log defaults to Exception Only. Delivery, Maintenance, Error, and Licensing all ship set to their most detailed logging tier by default, and Routing Rules defaults to Normal rather than Exception Only. This is intentional — delivery and error logs are the ones administrators reach for most often when a message goes missing or a feature misbehaves, so SmarterMail keeps them verbose from the start rather than requiring an administrator to remember to turn them up after the fact.

Process Logging can help administrators in a number of ways. For example:

Delivery Logs can help find out what happened to a particular message: if it was delivered, if it was delivered but rejected due to spam rules, whether it was moved based on a content filter, etc.
SMTP Logs can show why a message was rejected by the recipient's mail server.
Administrative Logs can show when a setting was changed, and which system administrator made the change.

The following log file types can be adjusted:

Administrative - The log level for any changes and/or modifications made by system administrator accounts.
Calendars - The log level for calendar appointments.
Certificates - The log level for SSL/TLS certificate activity, including automatic renewals (such as Let's Encrypt) and certificate binding to domains and services. Useful when a certificate fails to renew or a binding doesn't pick up a newly-issued certificate.
Content Filtering - The log level for any changes made due to Content Filtering rules.
Delivery - The log level for message delivery and spool operations.
Error - The log level for capturing any generic errors returned by SmarterMail.
Events - The log level for event sessions put in place for the system or user.
EWS Retrieval - The log level for EWS/Exchange retrieval sessions, used when SmarterMail pulls mail from an Exchange server. (This also logs information for Exchange-based mailbox migrations.)
Folder Auto-Clean - The log level for any folder auto-clean rules in place for the system or user.
General Errors - A broader error log, separate from the Error log above, that captures additional unhandled or unexpected errors surfaced by the application. If the Error log doesn't show what you're looking for, check this log as well.
IMAP Retrieval - The log level for IMAP retrieval sessions. (This also logs information for generic IMAP-based mailbox migrations, e.g., from Gmail or another IMAP-compatible provider.)
Indexing - The log level for SmarterMail indexing.
Licensing - The log level for any Licensing issues, such as activation issues.
Mailing Lists - The log level for items pertaining to Mailing Lists.
Maintenance - The log level for maintenance tasks performed by SmarterMail.
Message-ID - The log level for logging Message-ID's of all messages sent to mailing lists.
POP Retrieval - The log level for POP retrieval sessions. (This also logs information for mailbox migrations.)
Routing Rules - The log level for processing Routing Rules.
Spam Checks - The log level for all Spam Checks set up and in use.

Note: Two log types that appeared in older versions of this documentation — IIS and Conversion — no longer exist as configurable log levels in current builds of SmarterMail and have been removed from the list above. IIS-related issues are now best diagnosed through the Error and General Errors logs, or through the Windows Event Log on the host server.

Note: More detailed logs require more disk space. If you choose a detailed log, you may want to enable the auto-delete setting on the Log Files tab.

Performance Tuning

Performance Tuning allows a system administrator to manage some settings that may cause an increase in server utilization of memory and CPU. For example, there are several settings surrounding search indexing. Search indexing allows users to instantly find files in their mailbox, including messages, attachments, appointments, contacts, tasks, or notes. Following the initial scan of the server, SmarterMail continually monitors each user's mailbox for changes and then updates the index accordingly. This method of indexing reduces server utilization while increasing the speed with which search results are returned. However, especially on larger installations, search indexing can cause an increase in memory usage. Use this section to adjust the indexing configuration and cache size for your server:

Max Indexing Threads - The maximum number of threads to use for search indexing. Increasing this value will cause SmarterMail to use more CPU, but will allow the system to simultaneously index more users. By default, this value is set to 2 less than the server's logical processor count (reserving a couple of cores for the rest of the mail server's workload). For example, if your server has 32 processors, this value will default to 30. SmarterMail always enforces a floor of 1 thread, so on a single- or dual-core server this setting will simply default to 1 rather than 0 or a negative number.
Items to Index Per Pass - The number of items to index per user per index attempt. Increasing this number will increase memory usage and decrease the time it takes to index one user. However, it will increase the length of time it takes to index many small users if there are a few large users. (By default, this value is set to 2500.)
Seconds In Queue Before Indexing - The amount of time a user must be in the indexing queue before being indexed. This setting provides a buffer for many changes to a mailbox to ensure the same user is not indexed multiple times. Increasing this number will cause search results to be delayed further, but will result in indexing heavier users less frequently. (By default, this value is set to 60.)
OpenTelemetry Endpoint - This is the URL of the OpenTelemetry collector/receiver set up to receive data from SmarterMail. Generally, this is in an HTTPS format.
Config File Cache Size - This is what stores all JSON configuration files (domain settings, user settings, etc.) in memory so they don't need to be re-read from disk on every request. By default, this is set to Automatic, which currently resolves to 500 MB; if you manually set a value, SmarterMail enforces a floor of 50 MB (any lower value you enter will be raised back to 50 MB). This setting should only be changed when suggested by SmarterTools Support — for example, on a very large multi-tenant server with thousands of domains, Support might recommend raising this value if they see evidence of excessive disk I/O from configuration files being repeatedly reloaded.
Track Config File Cache Statistics - This is disabled, by default, as it can cause a decrease in performance due to CPU usage. As with the Config File Cache Size, it should only be enabled — temporarily — when suggested by SmarterTools Support. Enabling this will start tracking cache statistics in a performance counter on the system which can then be used to determine if a change in File Cache Size should be made.

View Logs

Use this section to quickly view the server's log files. Viewing a server's log files, especially when it's possible to narrow down the type of server action or protocol that is being viewed, allows system administrators to look for any specific errors that could cause reliability issues on the server or narrow down reasons why a specific behavior is being seen. For example, system administrators can review SMTP logs to see if an email was delivered or check ActiveSync logs to see if they can narrow down synchronization issues between a specific user's mailbox and their mobile device.

When viewing the SmarterMail logs, the following search strings will be available:

Start and End - The start and end dates for the log files you want to view.
Type - The type of log file that you would like to view.
Search - Type the words or phrases should be contained in the log files that SmarterMail returns.
Filter - When searching the logs, you can choose whether to display only lines that match the search definitions or to display related traffic as well. Change this selection from Only Matching Rows to Display Related Traffic in order to display extra data that occurred within the same session.

To search for a specific log, complete the date range, select the log type, and enter a search string. Then click Search. Any matching log files will be displayed.

Note: SmarterMail will only display up to 2MB of any specific log in the browser. If your search matches more data than that, the results will be truncated (oldest matching lines dropped first) — use Download below to retrieve the complete, untruncated log file instead.

Only Matching Rows vs. Display Related Traffic is worth understanding in detail, since it changes what SmarterMail actually returns. Every line written to a protocol log begins with a session identifier in brackets — for example, [26276072] — that ties every line generated during that single connection together. With Only Matching Rows selected, SmarterMail returns only the individual lines that contain your search text. With Display Related Traffic selected, SmarterMail first finds every session ID associated with a matching line, then returns every line in the log that shares that session ID — effectively reassembling the entire connection or conversation the matching line was part of, not just the one line that happened to match.

For example, suppose a user reports that their email client keeps disconnecting during an IMAP folder sync. On the View Logs tab, set Type to IMAP, enter the user's email address as the Search term, and select Display Related Traffic. Instead of seeing only the isolated line where the user's address appears (typically just the login line), you'll see the user's entire IMAP session from connection through authentication, folder selection, command exchange, and disconnect — which is usually what you need to determine whether the client disconnected cleanly, timed out, or was dropped by the server. This same technique works well combined with Protocol Logging: if IMAP logging is still set to Exceptions Only and the session in question didn't generate an error, temporarily raise IMAP logging to Detailed, reproduce the issue, then use Display Related Traffic to pull the full session once it's captured.

To download the entire log file in a .zip format — NOT just search results — click on Download. This allows you to get quick access to a domain's entire log file so that it can be reviewed more thoroughly on a local machine. If you only need the search results, click on Copy to Clipboard to copy the results to your clipboard, then past those results into your favorite text editor. (We recommend Notepad ++)

Services

Use this section to enable and/or disable specific services on the mail server. Generally, all of these services should be enabled. However, there are cases where an administrator may want to disable one or more. For example, a web host or ISP may want to limit users' access to incoming mail to POP only when they connect with an email client in order to conserve disk space on the mail server. In this case, the system administrator would want to stop the IMAP services. Another example would be a mail administrator for a large corporation who doesn't want users to add multiple email accounts and therefore read and reply to email from personal accounts as well as their corporate accounts. In this case, the administrator would want to disable the IMAP Retrieval and POP Retrieval services.

The following services can be enabled or disabled on the server:

EWS Retrieval - EWS retrieval is similar to IMAP Retrieval except it uses the EWS protocol for downloading messages from Exchange servers as well as from other SmarterMail servers.
IMAP - A client/server protocol in which email is received and held by the mail server. IMAP requires continual access to the client during the time that it is working with the mail server.
IMAP Retrieval - With IMAP retrieval, mail is retrieved from external IMAP servers (e.g., another mail server like GMail) and saved in a mailbox on the mail server.
Indexing - Indexes messages, contacts, calendars, tasks and notes so that users can search for specific mailbox items via the Web interface.
LDAP (Enterprise Edition Only) - A communication protocol for accessing online directory services. Programs like Outlook and Thunderbird use LDAP to retrieve contact lists from SmarterMail. SmarterMail will validate email addresses for user accounts, aliases, and mailing lists.
POP - An email protocol in which mail is saved in a mailbox on the mail server. When the end user reads the mail, it is immediately downloaded to the client computer and is no longer maintained on the mail server.
POP Retrieval - Similar to IMAP Retrieval, with POP retrieval, mail is retrieved from external POP3 servers and saved in a mailbox on the mail server.
SMTP - A TCP/IP (Internet) protocol used for sending and receiving email. However, since it's limited in its ability to queue messages at the receiving end, it's usually used with one of two other protocols, POP or IMAP, that let the user save messages in a server mailbox and download them periodically from the server. In other words, users typically use a program that uses SMTP for sending email and either POP or IMAP for receiving messages from their local server.
SpamFoo - The status of SpamFoo, the email classification and antispam client that runs locally.
Spool - The internal message queue used to deliver messages locally and to remote services.
XMPP (Enterprise Edition Only) - An open-source IM protocol designed to allow interoperability between different IM client programs. SmarterMail uses this protocol to power its chat functionality in the Web interface and/or third-party chat clients.

To modify the status of a service, select the desired service and click Start or Stop.

Note: Use extreme caution before stopping SMTP or Spool. Stopping SMTP means the server will stop accepting inbound mail entirely (senders will receive a connection-refused or timeout error rather than a bounce), and stopping Spool halts local and outbound delivery, meaning messages will simply queue up rather than send. Neither service being off is normally something you want in production — these are typically only stopped briefly while diagnosing a specific problem (for example, isolating whether a delivery issue is caused by the spool itself versus a downstream destination server) and should be restarted as soon as the diagnostic step is complete.

Mailbox Indexing

SmarterMail Search Indexing allows users to instantly find any files in the mailbox, including messages, attachments, appointments, contacts, tasks or notes. Following the initial scan of the server, SmarterMail continually monitors each user's mailbox for changes and updates the index accordingly. This method of indexing reduces server utilization while increasing the speed with which search results are returned.

Under the hood, indexing is a queue-based system, not a live filesystem watcher: when a mailbox changes (a new message arrives, an item is deleted, a calendar entry is updated, etc.), that user is placed into an indexing queue rather than being indexed immediately. Each user waits in the queue for the number of seconds configured in Seconds In Queue Before Indexing (60 seconds by default, as described in the Performance Tuning section above) before being processed, which prevents a mailbox that's being actively modified from being re-indexed dozens of times in a row. Once picked up, a user's status will typically progress through Queued, Indexing, and, for larger mailboxes that need index maintenance, Optimizing, before finally showing Completed.

System administrators can use this section to view the status of SmarterMail Search Indexing. Viewing the status of indexing can be beneficial when troubleshooting a problem. For example, if the mail service seems to be using a large amount of CPU, the system administrator can check to see if the cause of the temporary increase in CPU usage is due to indexing — if a large number of mailboxes are shown in the Indexing or Optimizing state at once (for example, right after a bulk import or a large mailbox migration), that's a strong sign that indexing, rather than mail flow, is the source of the load.

Mailbox Migrations

SmarterMail's Mailbox Migration tool makes it easy for users to switch email providers by giving them the ability to import emails, contacts, calendars, tasks, and notes to SmarterMail from most third-party mail servers. Under the hood, migrations run over one of two retrieval paths: a generic IMAP connection (covering Gmail, Yahoo, and virtually any other IMAP-compliant mail service) or an Exchange/EWS connection (covering on-premises Exchange 2007 SP1 and later, as well as Microsoft 365/Exchange Online). Whichever path is used, the same underlying retrieval engine that powers ongoing IMAP Retrieval and EWS Retrieval (see Process Logging, above) also drives one-time mailbox migrations.

That being said, users can do this on their own, with little input from a SmarterMail system administrator. While this normally is not an issue, there are times when an administrator may need to stop a migration altogether. That's where the Mailbox Migrations page comes in.

The following details can be seen for each entry in the list:

Email Address - The email address of the user performing the migration.
Status - The status of the migration being performed. The status displayed will be one of the following:

Queued - The migration was initiated and is waiting to start.
In Progress - The migration was started and is currently processing.
Completed - The migration is finished for that user.

To end the selected user's migration, select the user and click on the Actions (⋮) button and select Cancel Migration. The migration will be stopped, regardless of where it is in process. Canceling terminates the connection to the source server immediately, so no further items will be retrieved. However, administrators should be aware that canceling is not a database-style rollback: any messages, contacts, calendar items, or other data already copied into the SmarterMail mailbox before the cancellation remain there. In other words, a canceled migration stops making progress right away, but it will not automatically remove whatever was already imported up to that point — if a fully "clean" mailbox is required after a canceled migration, plan to manually review and remove any partially-imported items.

For more information on the Mailbox Migration process, including the fields necessary for different migration types, see the Migrating a Mailbox section of a User's Connectivity settings.

In addition, if there are issues with a migration, SmarterMail logs all migration activity. Therefore, a system administrator can check the Mailbox Importing logs for a user to see what happened, and find a resolution.