What's this?

This is a work in progress.

This entire site is about fixing problems with mail systems. There are many, and some of them have an impact on security. This site is concerned with the security issues.

This site contains tools to test mail systems, plus information about email security, how to find out more, and how to get problems fixed.

Why am I here?

You're probably here because you received an email Non-Delivery Report (NDR) containing a link to this page.

The Non-Delivery Report (NDR) says that when your mail service provider's mail server attempted (on your behalf) to send a message to us, our server rejected it. When it did so, our server replied to the sender, by means of the NDR, explaining the problem. Mail service providers sometimes, unfortunately, alter the NDR; some of them will even throw it away. They have their reasons, which may not always be admirable.

If this is why you're here, then there's a problem with your email configuration - even if everyone else is telling you that there isn't. But don't worry, it's easily fixed and it doesn't need to cost anything. We're not here to make a quick buck out of you.

Why did you reject my mail?

Probably because you told us to. Yes, really. Why else?

When our server inspected your mail (all mail servers do this) before deciding whether or not to accept it,
firstly it looked up the SPF records which your mail domain and/or your mail service provider have published, and then
secondly it checked that the server attempting to send the message is authorized by those records to send your mail.

If our server finds that the sending server is not authorized to send mail on your behalf, it will reject the message.

Our server will also reject mail (at least it's most likely to, but we can make exceptions) if an SPF record has one of certain very common but serious faults. Some people create their SPF records, it seems, by sheer guesswork. Sometimes their ingenuity is quite breathtaking. There are others who will, out of pure malice, try to abuse any and every facility available on the Internet to cause trouble for others. Deliberate falsification and misconfiguration of services, including services related to electronic mail, is amongst the techniques they use. We will have no truck with that.

If you wish you can send an email to the address in the NDR, which is a one-time address (good for use once only), and mail to that address will not be rejected. If the domain name in the NDR isn't the one you're sending your mail from, tell us the name in the NDR. That's very important. We will reply to explain the issue and what can be done about it.

But nobody else says that anything's wrong!

People often say "but I'm getting mail OK, and other people are getting mine, so it must all be working right and the problem must be at your end!" This flawed logic misses several points.

Tell the world

What matters is what is not delivered. By setting up a good SPF record you are taking a step to prevent forgery. Your published SPF record should say "mail from my domain only ever comes from servers on this list. Everything else is forged." But sometimes an SPF record is incorrectly formed, and sometimes it was perfectly correct when it was first deployed but it has been forgotten; then changes in the email system (for example a change of email provider) mean that it's no longer appropriate. Then the old record might well be telling the world, "all the mail that I send is forged!" The trouble then is that many receivers ignore your SPF record (perhaps they don't care about forgeries) and your mail isn't rejected by them. Then you send one to us, and it is rejected, so you think it must be our problem to fix.

Well no, unfortunately, it isn't.
It's yours.

Co-operation

Some of the security techniques in mail systems work
(a) only if the recipient server decides to apply them, so you can't test them by looking at what's delivered to recipients who don't apply them.
(b) Even then, mail delivery is prevented only in some circumstances. For example, SPF is intended to prevent the delivery of forgeries which are detected by the location on the network from where they were sent. It should make no difference, more or less, to the delivery of legitimate mail, but if your SPF record is set up incorrectly you can find legitimate mail from you is rejected.

Forgery

Forged mail on the public Internet is a very serious problem.

Criminals might be forging mail sent to you to look like it was sent by your boss, or your personal assistant, or your accounts department, or even by you.

In fact the greater part of the electronic mail on the Internet is from criminals, who forge mail messages by the billion.

People and organizations (even large ones, who you'd think ought to know better) have been duped out of millions of pounds by very simple forged emails. Google Inc., for example, paid more than a hundred million dollars to some guy in India who forged a few invoices. Eventually, they got quite a lot of it back.

Detecting forged mail

Fortunately, there are some very simple ways to detect forged mail.

Read the mail

One way to detect forged mail is to read it, carefully, and think about it. Many forged messages have very obvious indicators. Bad spelling and poor use of the language is one. Is the content even plausible?

Some criminals, however, know what they're doing and their messages are very well crafted indeed. Even after a very careful reading by someone familiar with the alleged writer, it might not be at all obvious that the message is forged.

Inspect the mail headers

Another way to detect forgery is to look at the mail headers.

The structure of a mail message

An electronic mail message is made up of two parts. They are the headers, of which there can be quite a few (nobody ever sees most of them), and a body. There's just one body. It can be empty, and although it's just about possible to imagine how an empty forged body might be malicious, you probably aren't worried about that. Some headers have a strictly defined structure, some don't, and most headers can be present or not present - but a message cannot be delivered without any headers. If you're at risk of being targeted by forged mail (and in 2018 that means most of us), it's worth learning a little about mail headers.

Viewing the mail headers

Your mail client (the thing that you use to read your mail - it might be called Microsoft Mail, Outlook, Seamonkey, Thunderbird, whatever) will usually show you something about most parts of the body, plus some representation of a very few headers (for example the 'Date', 'From', and 'Subject' headers), but it won't ususally show you most of the rest. It probably could do if you were to ask it to, but
(a) email headers can be difficult to understand, because they're not really intended for the casual reader, and
(b) when the mail client displays the headers to you, very often it makes a complete dog's breakfast of them, and
(c) headers can be forged, and because a lot of email is not cryptographically signed it can be difficult to determine if any particular header is forged or not.

Automatic forgery tests

Automated forgery tests are best performed by the mail servers which receive mail on your behalf. Your mail client could do more or less the same tests, but it cannot reject mail - it can only do things like putting the mail in the spam bucket or flagging it in some way. When mail is rejected by a mail server the sender knows that it was not delivered. If the mail is accepted by the server, even if your mail client drops the message into the spam bucket, then the sender knows that he has sent mail to a valid email address and the address becomes more valuable. Whether you later read the message or not is irrelevant.

More about automating forgery detection can be found below, but first we need to discuss the techniques in commoon use.

Forgery detection

SPF

Sender Policy Framework (SPF) is designed to detect forgeries. In principle it's a very simple idea. As of March 2018 the specification (well, most of it) is in RFC7208. Despite very popular misconceptions, SPF does not directly address the 'spam' problem except insofar as spam might be forged.

DKIM

DKIM is quite a bit more complex than SPF, and it does quite a bit more than just detect forgeries. Messages which employ DKIM are signed (using various cryptographic techniques) either by the sender, the relay(s), or both. DKIM is intended to enable those who employ it to build a reputation database from results of the signature verification processes. This database can be used both for forgery detection and for example for spam filtering.

A very brief description of SPF, taking some liberties.

The owner of a domain decides which mail servers are allowed to send mail on behalf of the domain, and then publishes a list of their IP addresses. Any mail which does not come from one of those IP addresses can be treated with suspicion. This means that if some criminal sends mail claiming to be from your domain, but the server which is used to send the mail is not in your published list, then without further consideration the recipient's server can reject the message, send it to quarantine or to the spam bucket, or simply discard it. There's no need at all to agonize over whether you should get involved in a money laundering operation with somebody you've never met. If the message is in fact rejected then you'll probably never see it.

SPF is implemented by two things 'records' published in the Domain Name System (DNS) and software on mail servers.

DNS records

The records for SPF are published in just the same way that domain names like 'fixmymail.uk' are published. Each name is associated with some text. Anyone can ask for the text associated with a name at any time. That's what the DNS is there for.

A very brief description of DKIM, taking more liberties.

To be continued...

Setting up a mail server to perform automatic forgery tests

If the sending and receiving mail systems are set up in the right way, they can make the detection of forgery very straightforward and completely automatic. The tools to do that have been available since the late 2000s and they are readily available. They are not trivial to install and use, and it must be left to mail providers to do most of the work. But if you own a domain name you can help. All you need to do is set up an SPF record. It's one line of text in the Domain Name System.

Forgery detection software

Unless you're running a mail server you don't need to worry about this part. If you are, you can use our software. It does a lot more than just verifying DKIM signatures and checking the mail envelopes against SPF records. Oh - yes, electronic mail has envelopes.

Unfortunately there is a great deal of misunderstanding about SPF, and there's a lot of half-baked nonsense published about it by well-meaning people who think they're helping.

One result of that is that in 2018 about one in three SPF records is broken in some way. Some are so badly broken that they cause rejection of legitimate mail. But they're easily fixed. That's why we're here. We are the SPF experts.