summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAaron Ball <nullspoon@oper.io>2019-02-16 15:07:52 -0700
committerAaron Ball <nullspoon@oper.io>2019-02-16 15:10:15 -0700
commit6c66a0ce19bc9b9a2c1f64a351fb9aae0e4b5d53 (patch)
treeeb4712e85fcc8e292851c90fbbcc0c0edd9486a1
parent711cc4a794cce2dcae6d711a990a3a4e5e5e5d70 (diff)
downloadoper.io-6c66a0ce19bc9b9a2c1f64a351fb9aae0e4b5d53.tar.gz
oper.io-6c66a0ce19bc9b9a2c1f64a351fb9aae0e4b5d53.tar.xz
Added Building a Reputable Mail Server
-rw-r--r--posts/building_a_reputable_mail_server.adoc256
-rw-r--r--posts/index.adoc2
2 files changed, 258 insertions, 0 deletions
diff --git a/posts/building_a_reputable_mail_server.adoc b/posts/building_a_reputable_mail_server.adoc
new file mode 100644
index 0000000..af6e60e
--- /dev/null
+++ b/posts/building_a_reputable_mail_server.adoc
@@ -0,0 +1,256 @@
+Building a Reputable Mail Server
+================================
+:author: Aaron Ball
+:email: nullspoon@oper.io
+
+
+Several months back, I moved my server to a different hosting provider and
+naturally received a different IP address. Since then, I have been struggling
+with one of the most prevalant problems one has when running their own mail
+server: building a good reputation to avoid spam directories. Since changing
+IPs, all of my emails to both Hotmail and Gmail have been going to spam,
+regardless of policy compliance.
+
+I did a [metric] truck load of research to remediate the situation and have
+finally resolved it. Most of the things I tried are relatively standard, but I
+realized during my research that there aren't very many good documents oriented
+to personal mail server owners. Most of the resources are aimed at bulk senders
+looking to accost users with [strikethrough]#spam# marketing emails. This is
+not me (and hopefully not you).
+
+To be clear before I proceed. This post is **not** about how to avoid spam
+filters so you can spam people. If you're spamming people, this guide will only
+help you for so long (and quite honestly, you can shove your spam where the sun
+don't shine). This post also won't go into the specifics of setting up a
+particular software like Exim or Postfix.
+
+This post is for the folks who might be interested in running their own servers
+for educational reasons. Or, perhaps you like to own your data. Maybe you hold
+the principle of freedom of the internet. You might also want to avoid
+mega-corporations. Or perhaps you may just have an affinity for tin-foil hats.
+
+Let's get to it!
+
+
+The Checklist
+-------------
+
+* <<_the_easy_stuff, The Easy Stuff>>
+** <<_static_ip, Static IP>>
+** <<_primary_hostname, Primary Hostname>>
+** <<_port_25_outbound, Port 25 Outbound>>
+** <<_have_accounts_with_major_providers, Have Accounts with Major Providers>>
+* <<_ip_reputation, IP Reputation>>
+* <<_spf_records, SPF Records>>
+* <<_dkim_signing, DKIM signing>>
+
+
+The Easy Stuff
+--------------
+
+I'll just cover these quickly since they are relatively simple.
+
+
+Static IP
+~~~~~~~~~
+
+If you understand how email works fundamentally (as in, you've read the SMTP
+link:https://tools.ietf.org/html/rfc821[RFC 821]), you know this isn't required
+by the spec. However, because of spammers (and other reasons), this is now an
+unofficial requirement to get emails through to the major providers.
+
+**TL;DR**: Don't run a mail server if you can't get a static IP. If you do, you
+will probably have big problems getting your emails delivered.
+
+**Detailed explanation**: Most providers won't deliver emails anywhere other
+than spam from IP addresses on which they cannot do
+link:https://en.wikipedia.org/wiki/Reverse_DNS_lookup[reverse DNS lookups].
+If you do this from a dynamic IP, you likely don't have control the PTR record
+and thus all of your emails will go straight to spam.
+
+The other reason you need a static IP is because major mail providers will
+often not deliver emails to inboxes unless the domain referenced in the _from_
+field has a DNS link:https://en.wikipedia.org/wiki/MX_record[MX record] that
+resolves to the IP address (your server) that sent the email. Because of
+minimum DNS TTLs, you can't [easily] get your DNS MX record to update fast
+enough when your IP changes for the major mail providers to correctly reconcile
+the new domain MX record with the sending IP (your server). This means when
+you're IP changes, even if you have automation updating DNS on the fly, you'll
+still have a window until the TTL expires for all of the various providers to
+re-query for the updated IP and approve delivery from it.
+
+
+Primary Hostname
+~~~~~~~~~~~~~~~~
+
+This is the name your mail server identifies as when connected to. If you want
+to see what this looks like, you can run `telnet <server> 25` to connect to a
+server's smtp service. The response you get back should say something like...
+
+ 220 <primary_hostname> ESMTP Exim 4.91 Sat, 09 Feb 2019 03:57:22 +0000
+
+NOTE: You probably cannot do this from your home internet, as most ISPs block
+port 25 outbound to prevent botnets from spamming the world. Marketing and
+current political issues aside, we haven't had actual net neutrality for years
+and this is one example of that fact, though it arguably does the internet some
+good.
+
+The primary hostname can actually be set to anything, as long as it is
+resolvable in DNS. We'll get into this a bit later with SPF records, but any
+mail server can send email on behalf any other domain. Whether that server is
+**permitted** to send email on behalf of that domain is another issue. We'll
+get more into the permissions aspect in the SPF records section. Regardless of
+all the flexibility you have here, just set this to the domain name from which
+you are sending email.
+
+
+Port 25 Outbound
+~~~~~~~~~~~~~~~~
+
+As mentioned in the last section, to send email you must be able to connect on
+port tcp/25, which most ISPs don't allow. If you can't do this for whatever
+reason, your emails will never reach their destination.
+
+
+Have Accounts with Major Providers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Troubleshooting any issues with a particular provider is very difficult without
+being able to see what your recipients are seeing. This is why I recommend you
+have an account with all of the major providers, so you can replicate any
+delivery issues your users might be having. Major providers you might consider
+having an account with are...
+
+* GMail
+* Hotmail/Live/Outlook
+* +++Mail.com+++
+* GMX
+* Proton Mail
+
+I personally have accounts with both Hotmail and Gmail, both of which I don't
+use for anything other than validating that I can send email to users of the
+respective services. I don't bother having an account with the other providers,
+since none of the people I email use those services (except Yahoo, but I've
+never had any delivery problems with them).
+
+
+
+
+IP Reputation
+-------------
+
+If you have an IP address that was previously used by spammers, it likely has a
+bad reputation. In an attempt to fight spammers, engineers created something
+called DNSBLs, or **DNS** **B**ased **L**ists. This is defined in the IETF
+link:https://tools.ietf.org/html/rfc5782[RFC 5782]. Effectively, an DNSBL can
+be used to define a whitelist or a blacklist of IPS and/or domains. If your IP
+address has a bad reputation, this is where you go to find out. Unfortunately
+however, there are hundreds of these available, and different providers use
+different lists, which makes reputation remediation a bit difficult.
+Fortunately, folks have built services to help you resolve the situation.
+Before I wrote link:/?p=checking-dnsbl-lists[my own script for this], I used
+link:https://mxtoolbox.com/blacklists.aspx[https://mxtoolbox.com] all the time.
+
+If your IP shows up as blacklisted on any of the providers, you should
+remediate that by their stated process. Each has their own process
+unfortunately, though most are relatively simple and quick. If you do not
+repair your server's IP reputation, all of your email's will be sent to spam by
+the major providers. For a better explanation of DNSBLs, see my post on
+link:link:/?p=checking-dnsbl-lists[checking dnsbl lists] (yes, I know that's
+redundant, like ATM machine).
+
+
+SPF Records
+-----------
+
+Here we have yet another spam-fighting mechanism, one with which I have no
+issues, unlike with DKIM. SPF stands for **S**ender **P**olicy **F**ramework.
+It uses DNS TXT records to public a list of IP address permitted to send emails
+on behalf of a given domain. These exist less as a whitelist and more as a sort
+of blacklist.
+
+Before SPF, any IP address could send email with from fields claiming to
+originate from any given domain. They still can do that, but now mail providers
+have a method with which they can verify permission to send emails on behalf of
+a given domain. I wrote link:/?p=Aol_Email_Hacked[a detailed post about this]
+back when AOL was "hacked" in 2014 if you want to read more details on SPF
+records and how not having them (or having them be too permissive) is
+dangerous.
+
+To better explain how these work, here is the spf record for my website
+(oper.io).
+
+----
+$ dig +short -t spf oper.io
+"v=spf1 mx a -all"
+----
+
+My spf [version 1] record permits any IP address that is pointed to by my MX or
+A records to send email on behalf of oper.io. In other words, emails with
+_...@oper.io_ in the _from_ field will be delivered field when the originating
+IP is in my MX or A records. Note though that I have a `-all` at the end of my
+SPF record. That is a hard deny for all other IP addresses. Other options are
+`+all`, which is a hard approval, and `?all`, which is a soft deny that
+effectively leaves the decision up to the mail server receiving the email.
+
+For most people running their own servers, the SPF record will likely only
+contain one IP. However, for businesses, these records have much greater
+application. Most businesses that send out marketing emails do not use their
+own mail servers to do so. They pay a third party and use SPF records to permit
+that third party to send emails on their behalf.
+
+
+DKIM Signing
+------------
+
+DKIM is _yet another_ method for validating email as coming from its specified
+source. DKIM stands for **D**omain**K**eys **I**dentified **M**ail. It is
+asymmetric cryptographic signing used to validate that an email came from the
+domain specified. To more easily describe this, I am going to simplify the
+architecture about which we are talking. I will assume that we have **one
+server**, and the DKIM selector is **default**.
+
+At a high level, your one server has an RSA key somewhere on it. Your email
+server is configured to sign all outbound email with the private key. This
+signature, along with some simple metadata, is embedded in your email using the
+**DKIM-Signature** header. Upon receipt, the service you emailed will read this
+header, and validate the signature by checking your public key, which you set
+up in DNS under a TXT record.
+
+NOTE: You can have more than one key doing singing (like one key per server),
+ and these keys are identified by something called a selector. The default
+ selector is called **default** (creative, I know).
+
+To check the domain signing key, you can do a dig command with...
+
+ dig +short -t txt default._domainkey.oper.io
+
+That won't return anything for my website as oper.io uses a randomly generated
+selector rather than default, which is set in my mail server configs. Were that
+to work though, the dig command would return something like...
+
+ $ dig +short -t txt default._domainkey.oper.io
+ v=DKIM1; k=rsa; p=MlGfMa03Rw+ZRaxSEjL67BkNZzsX8BPQ5tz11pN\010hivr...
+
+The retrieved public key is then used to validate the from and subject of the
+email with the signature in the header.
+
+While this spec isn't great, many major email providers expect it, so it's
+probably best to set it up to avoid being sent to spam.
+
+
+Summary
+-------
+
+Hopefully this post helps you if you are one of the rare hoomans who wishes to
+brave the perils of running your own email server. Though the task list to set
+up a reputable mail server is large at first, maintaining is is relatively
+simple once you have it.
+
+If you have any questions or would like me to write more about a particular
+topic, send me an email. It likely won't go to spam since my server only
+analyzes emails for smamminess based on content, rather than a long list of
+complex factors. :)
+
+[role="datelastedit"]
+Last edited: {docdate} {doctime}
diff --git a/posts/index.adoc b/posts/index.adoc
index 3c0022f..06b5eaa 100644
--- a/posts/index.adoc
+++ b/posts/index.adoc
@@ -7,6 +7,7 @@ Home
[role="index"]
New Posts
---------
+* link:?p=building_a_reputable_mail_server[Building a Reputable Mail server]
* link:?p=how_to_uninterest_me_in_your_job_opening[How to Uninterest Me in Your Job Opening]
* link:?p=why-linux-is-hard[Why Linux is Hard]
* link:?p=checking-dnsbl-lists[Checking DNSBL Lists]
@@ -64,6 +65,7 @@ Workstation
Server
~~~~~~
+* link:?p=building_a_reputable_mail_server[Building a Reputable Mail server]
* link:?p=checking-dnsbl-lists[Checking DNSBL Lists]
* link:?p=Managing_Linux_with_Linux[Managing Linux with Linux]
* link:?p=Exim_Spam_Filtering_with_Bogofilter[Exim Spam Filtering with Bogofilter]

Generated by cgit