Why am I here?

You're probably here because you received an email Non-Delivery Report (NDR) containing a link to this page.

The way that email works is that when you send an email to anyone, your mail always goes first to your email service provider. Then your service provider decides to what mail server it needs to go next, and sends it there, and so on. On its way to the destination, your mail can be passed through a long chain of servers.

If a mail message was sent to us, the last server in the chain is our server.

Like most mail servers, our server protects us from malicious and criminal mail. It's very strict, but it obeys the Internet rules. (Not all servers do that. In particular, some Microsoft servers handle SPF very badly, and their NDRs are next to useless.)

If our server rejects a message, it will always explain why in a Non-Delivery Report (*).

The Non-Delivery Report goes back directly to your mail provider's server.

If this is why you're here, then there's a problem with your email configuration - even if everyone else is telling you that there isn't. But it's probably easily fixed.

* Our NDR is for you, and it provides information which can be invaluable when tracing mail problems. Unfortunately, mail service providers sometimes alter the NDR; some of them will even throw it away, so you'll never even know of its existence - nor that your mail was not delivered. They have their reasons, which may not always be admirable.

Why did you reject my mail?

Probably because we were told to, by your SPF record.

Our server inspected your mail before deciding whether or not to accept it. Almost all mail servers do this.

* Firstly it looked up the SPF record which your mail domain and/or your mail service provider have published.

* Secondly it checked that the server attempting to send the message is authorized by that record to send your mail.

If the SPF record does not authorize the sending server to send your mail, our server will reject the message.

Our server will also reject mail (*) if an SPF record has one of certain very common but serious faults. Some people create their SPF records, it seems, by sheer guesswork. Sometimes their ingenuity is quite breathtaking. There are others who will, out of pure malice, try to abuse any and every facility available on the Internet to cause trouble for others. Deliberate falsification and misconfiguration of services, including services related to electronic mail, is amongst the techniques they use.

Co-operation

Some of the security techniques in mail systems work
(a) only if the recipient server decides to apply them, so you can't test them by looking at what's delivered to recipients who don't apply them.
(b) Even then, mail delivery is prevented only in some circumstances. For example, SPF is intended to prevent the delivery of forgeries which are detected by the location on the network from where they were sent. It should make no difference, more or less, to the delivery of legitimate mail, but if your SPF record is set up incorrectly you can find legitimate mail from you is rejected.

Forgery

Forged mail on the public Internet is a very serious problem.

Criminals might be forging mail sent to you to look like it was sent by your boss, or your personal assistant, or your accounts department, or even by you.

In fact the greater part of the electronic mail on the Internet is from criminals, who forge mail messages by the billion.

People and organizations (even large ones, who you'd think ought to know better) have been duped out of millions of pounds by very simple forged emails. Google Inc., for example, paid more than a hundred million dollars to some guy in India who forged a few invoices. Eventually, they got quite a lot of it back.

Detecting forged mail

Fortunately, there are some very simple ways to detect forged mail.

Read the mail

One way to detect forged mail is to read it, carefully, and think about it. Many forged messages have very obvious indicators. Bad spelling and poor use of the language is one. Is the content even plausible?

Some criminals, however, know what they're doing and their messages are very well crafted indeed. Even after a very careful reading by someone familiar with the alleged writer, it might not be at all obvious that the message is forged.

Inspect the mail headers

Another way to detect forgery is to look at the mail headers.

The structure of a mail message

An electronic mail message is made up of two parts. They are the headers, of which there can be quite a few (nobody ever sees most of them), and a body. There's just one body. It can be empty, and although it's just about possible to imagine how an empty forged body might be malicious, you probably aren't worried about that. Some headers have a strictly defined structure, some don't, and most headers can be present or not present - but a message cannot be delivered without any headers. If you're at risk of being targeted by forged mail (and in 2018 that means most of us), it's worth learning a little about mail headers.

Viewing the mail headers

Your mail client (the thing that you use to read your mail - it might be called Microsoft Mail, Outlook, Seamonkey, Thunderbird, whatever) will usually show you something about most parts of the body, plus some representation of a very few headers (for example the 'Date', 'From', and 'Subject' headers), but it won't ususally show you most of the rest. It probably could do if you were to ask it to, but
(a) email headers can be difficult to understand, because they're not really intended for the casual reader, and
(b) when the mail client displays the headers to you, very often it makes a complete dog's breakfast of them, and
(c) headers can be forged, and because a lot of email is not cryptographically signed it can be difficult to determine if any particular header is forged or not.

Automatic forgery tests

Automated forgery tests are best performed by the mail servers which receive mail on your behalf. Your mail client could do more or less the same tests, but it cannot reject mail - it can only do things like putting the mail in the spam bucket or flagging it in some way. When mail is rejected by a mail server the sender knows that it was not delivered. If the mail is accepted by the server, even if your mail client drops the message into the spam bucket, then the sender knows that he has sent mail to a valid email address and the address becomes more valuable. Whether you later read the message or not is irrelevant.

More about automating forgery detection can be found below, but first we need to discuss the techniques in commoon use.

Forgery detection

SPF

Sender Policy Framework (SPF) is designed to detect forgeries. In principle it's a very simple idea. As of March 2018 the specification (well, most of it) is in RFC7208. Despite very popular misconceptions, SPF does not directly address the 'spam' problem except insofar as spam might be forged.

DKIM

DKIM is quite a bit more complex than SPF, and it does quite a bit more than just detect forgeries. Messages which employ DKIM are signed (using various cryptographic techniques) either by the sender, the relay(s), or both. DKIM is intended to enable those who employ it to build a reputation database from results of the signature verification processes. This database can be used both for forgery detection and for example for spam filtering.

A very brief description of SPF, taking some liberties.

The owner of a domain decides which mail servers are allowed to send mail on behalf of the domain, and then publishes a list of their IP addresses. Any mail which does not come from one of those IP addresses can be treated with suspicion. This means that if some criminal sends mail claiming to be from your domain, but the server which is used to send the mail is not in your published list, then without further consideration the recipient's server can reject the message, send it to quarantine or to the spam bucket, or simply discard it. There's no need at all to agonize over whether you should get involved in a money laundering operation with somebody you've never met. If the message is in fact rejected then you'll probably never see it.

SPF is implemented by two things 'records' published in the Domain Name System (DNS) and software on mail servers.

DNS records

The records for SPF are published in just the same way that domain names like 'fixmymail.uk' are published. Each name is associated with some text. Anyone can ask for the text associated with a name at any time. That's what the DNS is there for.

Setting up a mail server to perform automatic forgery tests

If the sending and receiving mail systems are set up in the right way, they can make the detection of forgery very straightforward and completely automatic. The tools to do that have been available since the late 2000s and they are readily available. They are not trivial to install and use, and it must be left to mail providers to do most of the work. But if you own a domain name you can help. All you need to do is set up an SPF record. It's one line of text in the Domain Name System.

Forgery detection software

Unless you're running a mail server you don't need to worry about this part. If you are, you can use our software. It does a lot more than just verifying DKIM signatures and checking the mail envelopes against SPF records. Oh - yes, electronic mail has envelopes.

Unfortunately there is a great deal of misunderstanding about SPF, and there's a lot of half-baked nonsense published about it by well-meaning people who think they're helping.

One result of that is that in 2018 about one in three SPF records is broken in some way. Some are so badly broken that they cause rejection of legitimate mail. But they're easily fixed. That's why we're here. We are the SPF experts.