Building a Reputable Mail Server

Several months back, I moved my server to a different hosting provider and naturally received a different IP address. Since then, I have been struggling with one of the most prevalant problems one has when running their own mail server: building a good reputation to avoid spam directories. Since changing IPs, all of my emails to both Hotmail and Gmail have been going to spam, regardless of policy compliance.

I did a [metric] truck load of research to remediate the situation and have finally resolved it. Most of the things I tried are relatively standard, but I realized during my research that there aren’t very many good documents oriented to personal mail server owners. Most of the resources are aimed at bulk senders looking to accost users with spam marketing emails. This is not me (and hopefully not you).

To be clear before I proceed. This post is not about how to avoid spam filters so you can spam people. If you’re spamming people, this guide will only help you for so long (and quite honestly, you can shove your spam where the sun don’t shine). This post also won’t go into the specifics of setting up a particular software like Exim or Postfix.

This post is for the folks who might be interested in running their own servers for educational reasons. Or, perhaps you like to own your data. Maybe you hold the principle of freedom of the internet. You might also want to avoid mega-corporations. Or perhaps you may just have an affinity for tin-foil hats.

Let’s get to it!

The Checklist

The Easy Stuff

I’ll just cover these quickly since they are relatively simple.

Static IP

If you understand how email works fundamentally (as in, you’ve read the SMTP RFC 821), you know this isn’t required by the spec. However, because of spammers (and other reasons), this is now an unofficial requirement to get emails through to the major providers.

TL;DR: Don’t run a mail server if you can’t get a static IP. If you do, you will probably have big problems getting your emails delivered.

Detailed explanation: Most providers won’t deliver emails anywhere other than spam from IP addresses on which they cannot do reverse DNS lookups. If you do this from a dynamic IP, you likely don’t have control the PTR record and thus all of your emails will go straight to spam.

The other reason you need a static IP is because major mail providers will often not deliver emails to inboxes unless the domain referenced in the from field has a DNS MX record that resolves to the IP address (your server) that sent the email. Because of minimum DNS TTLs, you can’t [easily] get your DNS MX record to update fast enough when your IP changes for the major mail providers to correctly reconcile the new domain MX record with the sending IP (your server). This means when you’re IP changes, even if you have automation updating DNS on the fly, you’ll still have a window until the TTL expires for all of the various providers to re-query for the updated IP and approve delivery from it.

Primary Hostname

This is the name your mail server identifies as when connected to. If you want to see what this looks like, you can run telnet <server> 25 to connect to a server’s smtp service. The response you get back should say something like…​

220 <primary_hostname> ESMTP Exim 4.91 Sat, 09 Feb 2019 03:57:22 +0000
Note
 You probably cannot do this from your home internet, as most ISPs block port 25 outbound to prevent botnets from spamming the world. Marketing and current political issues aside, we haven’t had actual net neutrality for years and this is one example of that fact, though it arguably does the internet some good.

The primary hostname can actually be set to anything, as long as it is resolvable in DNS. We’ll get into this a bit later with SPF records, but any mail server can send email on behalf any other domain. Whether that server is permitted to send email on behalf of that domain is another issue. We’ll get more into the permissions aspect in the SPF records section. Regardless of all the flexibility you have here, just set this to the domain name from which you are sending email.

Port 25 Outbound

As mentioned in the last section, to send email you must be able to connect on port tcp/25, which most ISPs don’t allow. If you can’t do this for whatever reason, your emails will never reach their destination.

Have Accounts with Major Providers

Troubleshooting any issues with a particular provider is very difficult without being able to see what your recipients are seeing. This is why I recommend you have an account with all of the major providers, so you can replicate any delivery issues your users might be having. Major providers you might consider having an account with are…​

  • GMail

  • Hotmail/Live/Outlook

  • Mail.com

  • GMX

  • Proton Mail

I personally have accounts with both Hotmail and Gmail, both of which I don’t use for anything other than validating that I can send email to users of the respective services. I don’t bother having an account with the other providers, since none of the people I email use those services (except Yahoo, but I’ve never had any delivery problems with them).

IP Reputation

If you have an IP address that was previously used by spammers, it likely has a bad reputation. In an attempt to fight spammers, engineers created something called DNSBLs, or DNS Based Lists. This is defined in the IETF RFC 5782. Effectively, an DNSBL can be used to define a whitelist or a blacklist of IPS and/or domains. If your IP address has a bad reputation, this is where you go to find out. Unfortunately however, there are hundreds of these available, and different providers use different lists, which makes reputation remediation a bit difficult. Fortunately, folks have built services to help you resolve the situation. Before I wrote my own script for this, I used https://mxtoolbox.com all the time.

If your IP shows up as blacklisted on any of the providers, you should remediate that by their stated process. Each has their own process unfortunately, though most are relatively simple and quick. If you do not repair your server’s IP reputation, all of your email’s will be sent to spam by the major providers. For a better explanation of DNSBLs, see my post on checking dnsbl lists (yes, I know that’s redundant, like ATM machine).

SPF Records

Here we have yet another spam-fighting mechanism, one with which I have no issues, unlike with DKIM. SPF stands for Sender Policy Framework. It uses DNS TXT records to public a list of IP address permitted to send emails on behalf of a given domain. These exist less as a whitelist and more as a sort of blacklist.

Before SPF, any IP address could send email with from fields claiming to originate from any given domain. They still can do that, but now mail providers have a method with which they can verify permission to send emails on behalf of a given domain. I wrote a detailed post about this back when AOL was "hacked" in 2014 if you want to read more details on SPF records and how not having them (or having them be too permissive) is dangerous.

To better explain how these work, here is the spf record for my website (oper.io).

$ dig +short -t spf oper.io
"v=spf1 mx a -all"

My spf [version 1] record permits any IP address that is pointed to by my MX or A records to send email on behalf of oper.io. In other words, emails with …​@oper.io in the from field will be delivered field when the originating IP is in my MX or A records. Note though that I have a -all at the end of my SPF record. That is a hard deny for all other IP addresses. Other options are +all, which is a hard approval, and ?all, which is a soft deny that effectively leaves the decision up to the mail server receiving the email.

For most people running their own servers, the SPF record will likely only contain one IP. However, for businesses, these records have much greater application. Most businesses that send out marketing emails do not use their own mail servers to do so. They pay a third party and use SPF records to permit that third party to send emails on their behalf.

DKIM Signing

DKIM is yet another method for validating email as coming from its specified source. DKIM stands for DomainKeys Identified Mail. It is asymmetric cryptographic signing used to validate that an email came from the domain specified. To more easily describe this, I am going to simplify the architecture about which we are talking. I will assume that we have one server, and the DKIM selector is default.

At a high level, your one server has an RSA key somewhere on it. Your email server is configured to sign all outbound email with the private key. This signature, along with some simple metadata, is embedded in your email using the DKIM-Signature header. Upon receipt, the service you emailed will read this header, and validate the signature by checking your public key, which you set up in DNS under a TXT record.

Note
 You can have more than one key doing singing (like one key per server), and these keys are identified by something called a selector. The default selector is called default (creative, I know).

To check the domain signing key, you can do a dig command with…​

dig +short -t txt default._domainkey.oper.io

That won’t return anything for my website as oper.io uses a randomly generated selector rather than default, which is set in my mail server configs. Were that to work though, the dig command would return something like…​

$ dig +short -t txt default._domainkey.oper.io
v=DKIM1; k=rsa; p=MlGfMa03Rw+ZRaxSEjL67BkNZzsX8BPQ5tz11pN\010hivr...

The retrieved public key is then used to validate the from and subject of the email with the signature in the header.

While this spec isn’t great, many major email providers expect it, so it’s probably best to set it up to avoid being sent to spam.

Summary

Hopefully this post helps you if you are one of the rare hoomans who wishes to brave the perils of running your own email server. Though the task list to set up a reputable mail server is large at first, maintaining is is relatively simple once you have it.

If you have any questions or would like me to write more about a particular topic, send me an email. It likely won’t go to spam since my server only analyzes emails for smamminess based on content, rather than a long list of complex factors. :)

Last edited: 2019-09-28 23:22:07 UTC