Configuring SmarterMail for Failover (Windows Only)

Who Should Use This

This document is intended for use by administrators deploying SmarterMail in high-volume environments and/or for organizations that want to ensure maximum uptime. It provides minimal system requirements and considerations for deploying SmarterMail in a failover environment.

Failover Overview

SmarterMail Enterprise allows organizations to decrease the likelihood of service interruptions and virtually eliminate downtime by installing SmarterMail on a standby server that is available should the primary mail server suffer a service interruption. For businesses that use their mail server as a mission-critical part of their operations, failover functionality ensures that the business continues to communicate and that productivity remains at the highest levels possible, even if there is a primary server failure.

Understanding How Failover Works

The main components of failover functionality are; a primary server that acts as the default SmarterMail server and manages the licensing of the server cluster, and a secondary server that remains connected and available in a “hot standby” mode until the primary server experiences problems with network access or system hardware. It's considered a "hot standby" because the server is "hot", in that it's powered on, but the SmarterMail service (and possibly the web server) are inactive/disabled.

If the primary server fails, SmarterMail can be configured to automatically enable services on the secondary server. When this occurs, the secondary server then takes over responsibility for processing background threads and supporting all email functionality. This server will remain in active status until another failure occurs or the primary mail server comes back online.

The initial set up of SmarterMail's failover functionality entails system administrators manually disabling both the node and SmarterMail service on the primary server and then starting the node and SmarterMail service on the standby. However, system administrators can easily use third-party monitoring systems and script an automated failover and recovery strategy as needed. An example of this is provided at the end of this document.

Minimal System Requirements

• A minimum of two servers running Microsoft Windows Server 2019 64-bit. (Windows Server Core is not currently supported).
• Three IP addresses
• Both servers must have their server times synchronized
• A domain account or local system User or Group account with bidirectional authentication.
Note: SmarterMail can NOT be run using Local System, Local Service or Network Service in a failover configuration.
• NFS/SMB share for mail and system files. We recommend that the share is running on a NAS/SAN that is configured as RAID 10

Adding Network Load Balancing to Your Servers

1. Open the server manager console
2. Right click on Features in the tree view and select Add Features
3. Check the box next to Network Load Balancing and select Next
4. Click Install
5. Once the installation finishes, click Close

Configuring the Load Balanced Cluster for Use with Failover

1. Navigate to Start → Administrative Tools → Network Load Balancing Manager
2. Click the Cluster menu item and select New
3. In the New Cluster: Connect window, type the IP of your primary server in the Host: text box and select New
4. When the Interface Name and Interface IP appear, select the Interface Name and click Next
5. Since this is the primary node, ensure the host Priority is set to 1
6. In the New Cluster: Host Parameters window, confirm the IP address and Subnet mask are correct and change the initial host state to Stopped. This is to prevent any issues with connectivity if a machine randomly reboots or suffers from a hardware failure. If all nodes are set to Started for their initial host state, traffic will be split between the two (or more) machines.
Note: Monitoring software can be used to execute scripts that will start and stop hot standbys in the event of a failure and recovery. If you are not executing scripts via monitoring software then all failover will need to be handled manually.
7. Click Next
8. In the New Cluster: Cluster IP Addresses window, click Add and enter in your cluster IP address and the same subnet mask as in Step 6
9. Select Next
10. In the New Cluster: Cluster Parameters window, confirm the IP address and subnet mask, then enter a Full Internet Name, though this is optional
11. Ensure the cluster operation mode is set to Multicast
12. Click Next
13. In the New Cluster: Port Rules window, click Edit
    • If you want you can restrict the cluster IP to work on an individual port or across a port range. You can also simply allow the cluster IP to work across all ports on the server
    • Ensure your port rules are set to Single Host in the Filtering Mode section
14. Click OK
15. Verify your settings and click Finish to complete the setup

Joining Additional Nodes to the Cluster

1. From the secondary server navigate to Start → Administrative Tools → Network Load Balancing Manager
2. Click the Cluster menu item and select Connect to Existing.
Note: the existing cluster will need to be running before a secondary node can be added
3. In the Connect to Existing: Connect window, enter the IP address of your existing cluster as the Host and click Connect
4. Select the existing cluster that appears in the Clusters section and click Finish
5. In the main Network Load Balancing Manager, expand Network Load Balancing Clusters and right-click on your Cluster (it may be the IP address of your cluster) and select Add Host to Cluster
6. In the Add Host to Cluster: Connect window, enter the IP address of the secondary server in the Host: section and click Connect
7. When the Interface Name and Interface IP appear, select the Interface Name and click Next
8. In the Add Host to Cluster: Host Parameters window, confirm the IP address and subnet mask and ensure the Initial Host State is set to Stopped. As this is the second node you're adding to your cluster, the Priority should be set at 2
9. Click Next
10. Just as with the primary node, in the Add Host to Cluster: Port Rules window you have the ability to set this node to respond via specific ports or a port range. If you wish to set these rules, click Edit. Otherwise, click Finish to complete the setup
11. Wait for the nodes to converge and, if necessary, stop the secondary sever by right-clicking the second server's name, select Control Host → Stop

Configure a Shared Service Directory

1. Using Network File Sharing (NFS) or Samba (SMB), create a shared directory named SmarterMail, preferably on a NAS or SAN.
Note: We recommend that this shared directory be hosted on a server that utilizes a RAID 10 configuration for the data.
2. Inside that new SmarterMail folder, create a Settings folder
3. Configure your permissions accordingly. The SmarterMail service needs to run as a domain account or a local account with bidirectional authentication. You can configure this within the Windows Services console. When running SmarterMail with failover, Local System, Local Service and Network Service users are not allowed.
Note: When performing updates to the software, the credentials will need to be re-applied to the service

Configuring a Fresh Installation of SmarterMail for Failover

1. Manually install and configure a primary SmarterMail server using the .MSI file available from the SmarterMail downloads page. Then, stop the service on this primary installation.
2. Manually install another SmarterMail Enterprise instance on a second server. This new installation will be your standby. Leave all setup information as the default settings and after setup is complete, configure SmarterMail as an IIS site.
3. Stop the SmarterMail service on the standby
4. Edit the failover.json file in the primary server's Settings folder as follows.
• FailoverIPAddress - Set this to the IP address of the Network Load Balancer
• IsEnabled - Set this to True
• SharedSystemFilesPath - Set to the shared network shared system folder

A sample failover.json would look like this (this example is for a Windows server):

                    {

                    "NodeId": "a51eba87-c8c6-49e3-812f-84e46ab617e7",

                    "FailoverIPAddress": "122.32.55.241",

                    "IsEnabled": true,

                    "SharedSystemFilesPath": "\\\\serverName\\SmarterMail\\Service\\Settings"

                    }
                

The code should look like the above: casing, proper escaping of paths, etc. in order for the JSON to be read properly.

5. Save this file, then copy it to the standby's Settings folder, replacing the existing failover.json
6. Copy over all folders and files from SmarterMail's Settings folder to the Settings folder in the shared service directory you created.
7. Start the service on the standby server and verify that the paths are pointing to the network shared paths
8. Activate your Enterprise key on the standby by logging into SmarterMail's management interface as the system administrator and going to the activation section. Then stop the SmarterMail service on the server
9. Start the service on the primary server, then reactivate your Enterprise license key in the SmarterMail management interface
10. After re-activating the license, go to IP Addresses and bind all the ports to the load balancer's IP address and make sure no other IPs have any ports bound to them
11. Both servers are now set up for failover. To verify this, log into the primary server as the system administrator and go to Gateways / Failover. The servers that are part of the failover cluster will be displayed on the Failover Servers tab.

Adding Failover to an Existing Installation of SmarterMail

1. Ensure the primary server is running the latest version of SmarterMail and that it is also configured as an IIS site. Ensure the IIS binding is pointing to your cluster IP address
2. Install SmarterMail on a standby and configure it as an IIS site. Ensure the cluster node is stopped on the standby and ensure the IIS binding is also pointing to the cluster IP
3. Stop the SmarterMail service on the standby
4. Copy all of your mail data (located in C:\SmarterMail\ by default) to your shared service directory. If possible, use robocopy to do this because it will not result in any downtime for the mail service
5. Once robocopy finishes, run it one more time. This second pass will only copy any new data
6. Stop the SmarterMail service on the primary server
7. Edit the failover.json file in the primary server's Settings folder as follows:
• FailoverIPAddress - Set this to the IP address of the Network Load Balancer
• IsEnabled - Set this to True
• SharedSystemFilesPath - Set to the shared network shared system folder

A sample failover.json would look like this:

                    {

                    "NodeId": "a51eba87-c8c6-49e3-812f-84e46ab617e7",

                    "FailoverIPAddress": "122.32.55.241",

                    "IsEnabled": true,

                    "SharedSystemFilesPath": "\\\\serverName\\SmarterMail\\Service\\Settings"

                    }
                

The code should look like the above: casing, proper escaping of paths, etc. in order for the JSON to be read properly. Also, due to size limitations, in the sample above the SharedSystemFilesPath is split across 2 lines -- that should be ONE line.

8. Copy that failover.json file, after you've edited it, and move it to the same location on the standby. You should replace the file on the standby, if it already exists.
9. Run the robocopy one more time to copy over any modified files and remaining spool emails
10. Copy over all folders and files from the SmarterMail service's Settings folder to the Settings folder in the shared service directory you created.
11. Edit the domains.json file in the shared Settings folder and change the path of your domains to match the new NFS\SMB path. (For example, \\NAS01\SmarterMail\Domains\mydomain.com)
12. Edit the settings.json file and replace any instances of the old physical path's with your new network location for SmarterMail. (For example, if all of your data was hosted on E:\SmarterMail, you would then perform a find and replace for all instances of E:\SmarterMail to \\NAS01\SmarterMail).
13. On the primary server, go to Start → Administrative Tools → Network Load Balancing Manager and stop the cluster node, then start the NLB on the secondary node
14. Start the SmarterMail service on the standby
15. Access SmarterMail's web interface at the cluster IP and sign in as the system administrator
16. Activate your Enterprise key on the standby by logging into SmarterMail's management interface as the system administrator and going to the Licensing section.
17. Verify that the data and settings are being picked up from the shared Service directory
18. Stop the SmarterMail service on the standby and stop the secondary cluster node
19. Start the cluster node and the SmarterMail service on the primary server
20. Sign into the web interface on the primary server and re-activate the Enterprise license key by going to the Licensing section.
21. Verify mail data and settings are being accessed from the shared service directory

Scripting Failover

Below is an example of a PowerShell script that can be created to automate the SmarterMail failover process. You can utilize a third party monitoring product such as PRTG or SolarWinds (though there are many others) to execute this script when a failure is detected.

Prepping PowerShell on the Servers

The servers will need to be configured to run remote scripts and accept remote PowerShell sessions. Therefore, on each server, run the following commands within an elevated PowerShell console:

• Set-ExecutionPolicy RemoteSigned - Press Y to accept
• Enable-PSRemoting -force

Sample Script - Stop a Primary Server and Start the Hot Standby

In the scripts below, replace the “WAN” variable called in the –hostname parameter with the name of your interface. This can be obtained by opening a PowerShell console on the server and typing Get-NlbClusterNodeNetworkInterface. Also replace Server01 and Server02 with the NetBIOS names of your servers.

$StopPrimary = New-PSSession -ComputerName Server01 
Invoke-Command -Session $StopPrimary -ScriptBlock { Import-Module NetworkLoadBalancingClusters ; Stop-nlbclusternode -HostName Server01 -InterfaceName "WAN" ; 
import-module WebAdministration ; stop-webapppool SmarterMail; set-service -computerName Server01 -name mailservice -status stopped ; remove-pssession Server01}

$StartSecondary = New-PSSession -ComputerName Server02 
Invoke-Command -Session $StartSecondary -ScriptBlock { Import-Module NetworkLoadBalancingClusters ; Start-nlbclusternode -HostName Server02 -InterfaceName "WAN" ;
set-service -computerName Server02 -name mailservice -status running ; import-module WebAdministration ; start-webapppool SmarterMail ; remove-pssession Server02 }

Sample Script - Stop the Hot Standby and Re-start the Primary Server

These scripts can be used to bring the primary server back online and stop the standby after your monitoring software issues an all-clear.

$StopSecondary = New-PSSession -ComputerName Server02 
Invoke-Command -Session $StopSecondary -ScriptBlock { Import-Module NetworkLoadBalancingClusters ; Stop-nlbclusternode -HostName Server02 -InterfaceName "WAN" ; 
import-module WebAdministration ; stop-webapppool SmarterMail; set-service -computerName Server02 -name mailservice -status stopped ; remove-pssession Server02}

$StartPrimary = New-PSSession -ComputerName Server01 
Invoke-Command -Session $StartPrimary -ScriptBlock { Import-Module NetworkLoadBalancingClusters ; Start-nlbclusternode -HostName Server01 -InterfaceName "WAN" ;
set-service -computerName Server01 -name mailservice -status running ; import-module WebAdministration ; start-webapppool SmarterMail ; remove-pssession Server01 }