Mail Server on Synology with MailPlus

2018-12-23 · synology, mail, mailplus

I know, running your own mail server is unfashionable, and I've had the "never again" talk with myself more than once. Deliverability lives and dies on DNS and reputation, and one bad setting makes you an open relay. Still, for a small office that wants mail on its own hardware, Synology MailPlus Server on DSM 6 is a surprisingly complete package: Postfix and Dovecot underneath, spam filter, antivirus, a web client and mobile apps, five free accounts and paid licenses beyond that. Here's how I bring one up on a DS918+ for my own domain.

Before you start, check the boring requirements. You need a static public IP with a PTR record your ISP will set (most big providers reject mail without one), outbound port 25 not blocked (many residential lines do), a domain whose DNS you control, and a valid certificate for the mail hostname. Then install MailPlus Server and MailPlus from Package Center.

DNS records first

Before touching the NAS, publish the records. Example for example.com with the NAS at 203.0.113.10:

mail.example.com.      A     203.0.113.10
example.com.           MX 10 mail.example.com.
example.com.           TXT   "v=spf1 mx -all"
_dmarc.example.com.    TXT   "v=DMARC1; p=quarantine; rua=mailto:postmaster@example.com"
10.113.0.203.in-addr.arpa.  PTR  mail.example.com.   (set by your ISP)

The DKIM record comes later, after MailPlus generates the key.

Setup wizard

Open MailPlus Server. The wizard asks:

Domain name:    example.com
Account type:   Local users (or Domain users if the NAS is joined to AD/LDAP)
Hostname (FQDN): mail.example.com
Network interface: the LAN interface

Choose Create a new setup. It configures SMTP on 25 and enables the local delivery. I then go to Service → SMTP and set Maximum size per email to 25 MB and tick Enable SMTP authentication, which is required for users to send.

Secure protocols and port forwards

Service → IMAP/POP3: enable IMAP and IMAP SSL/TLS, leave POP3 off unless someone needs it. Service → SMTP → Advanced: enable SMTP-SSL (465) and SMTP-TLS (587). Assign the certificate under Control Panel → Security → Certificate → Configure, where MailPlus Server appears as a service.

Then forward ports on the router:

TCP 25   -> NAS (incoming mail)
TCP 465  -> NAS (submission, SSL)
TCP 587  -> NAS (submission, STARTTLS)
TCP 993  -> NAS (IMAPS)

Leave 143 and 110 closed.

DKIM signing

Security → Authentication → tick Enable DKIM signing on outbound mails, set a selector (mail), click Generate public key. Copy the key into DNS:

mail._domainkey.example.com.  TXT  "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA..."

On the same page enable SPF verification and DMARC verification for inbound mail so forged mail to your users gets flagged.

Spam and antivirus

Security → Spam: enable, set the threshold to 5.0 initially (I'd rather tighten it later than have users miss real mail), and tick Learn spam automatically. Users mark spam in MailPlus and the Bayes filter learns from it. Security → Antivirus: enable with ClamAV (free) and daily definition updates. Security → Content Scan can block executable attachments, which I recommend.

Mailboxes and aliases

Users are DSM users; give the staff group MailPlus privileges under Control Panel → Privileges. Then MailPlus Server → Account → Activate the accounts you want to use (this is where the license count matters). Aliases live under Account → Alias:

info@example.com        -> jsmith, akim
postmaster@example.com  -> admin-user

Also set Domain → Catch-all if you want unknown recipients delivered somewhere instead of bounced. I do not; it attracts spam.

Send yourself a test

Send a mail from a MailPlus account to a Gmail address and view the original message headers. It should show:

spf=pass
dkim=pass header.i=@example.com
dmarc=pass

Then test from outside that you are not an open relay:

telnet mail.example.com 25
EHLO test
MAIL FROM:<x@other.org>
RCPT TO:<y@another.org>

You must get 554 5.7.1 <y@another.org>: Relay access denied. MailPlus Server → Queue and → Log show delivery status for every message.

What usually goes wrong

Outgoing mail bounces with "550 5.7.1 [203.0.113.10] The IP you're using to send mail is not authorized" or a Spamhaus PBL listing: the IP is in a residential range. Use the Service → SMTP → Relay option to relay through your ISP's or a transactional provider's SMTP server with authentication.
Gmail marks everything as spam although SPF and DKIM pass: the PTR record is missing or does not match the EHLO hostname. Set the PTR and make sure Hostname in MailPlus is mail.example.com.
Users cannot send from Outlook: "The server rejected the sender address" means SMTP authentication is off, or the account is not activated in MailPlus Server.
Log fills with "warning: hostname ... does not resolve to address" entries: normal for scanners hitting port 25; enable Security → Auto Block on the MailPlus page to slow them down.

The package is the easy part. If mail goes to spam, it's DNS, nine times out of ten.