How to send emails and avoid them being classified as spam?

Question

This is a canonical question about how to handle email sent from your server being misclassified as spam. For additional information you may find these similar questions helpful:

• Best Practices for preventing you from looking like a spammer
• Fighting Spam - What can I do as an: Email Administrator, Domain Owner, or User?

Sometimes I want to send newsletters to my customers. The problem is, that some of the emails get caught as spam messages. Mostly by Outlook at the client (even in my own Outlook 2007).

Now I want to know what should be done to create "good" emails. I know about reverse lookup etc., but (for example), what about a unsubscribe link with an unique ID? Does that increase a spam rating?

Answer 1

Be sure that your emails don’t look like typical spam emails: don’t insert only a large image; check that the character-set is set correctly; don’t insert “IP-address only” links. Write your communication as you would write a normal email. Make it really easy to unsubscribe or opt-out. Otherwise, your users will unsubscribe by pressing the “spam” button, and that will affect your reputation.

On the technical side: if you can choose your SMTP server, be sure it is a “clean” SMTP server. IP addresses of spamming SMTP servers are often blacklisted by other providers. If you don’t know your SMTP servers in advance, it’s a good practice to provide configuration options in your application for controlling batch sizes and delay between batches. Some mail servers don’t accept large sending batches or continuous activity.

Use email authentication methods, such as SPF, and DKIM to prove that your emails and your domain name belong together. The nice side-effect is you help in preventing that your email domain is spoofed. Also check your reverse DNS to make sure the IP address of your mail server points to the domain name that you use for sending mail.

Make sure that the reply-to address of your emails are a valid, existing addresses. Use the full, real name of the addressee in the To field, not just the email-address (e.g. "John Doe" <[email protected]> ) and monitor your abuse accounts, such as [email protected] and [email protected].

Answer 2

Automatically unsubscribe recipients of your message whose e-mail addresses bounce, and establish complaint feedback loops with major mail providers and automatically unsubscribe recipients who report your message as spam/junk. This will go a long way to improving your reputation and deliverability.

Answer 3

This question mentions that the basics are in place, but as we're pointing others to this as a Canonical Question I just want to be sure we cover our bases.

These minimums are essentially required these days:

1. Make sure you have forward and reverse DNS configured correctly. A mail server has to identify itself in a HELO/EHLO exchange, that name should lookup to the IP the server is using. Similarly the reverse lookup of that IP should return the name.

2. Make sure your server is actually sending the hostname in that handshake. Your server should not be sending an IP address.

3. Make sure your IP address isn't on any DNSRBLs (blacklists). If it is, get that taken care of.

4. Check the reputation of your IP with the more popular reputation services (SenderScore is a big one right now, but that might not hold up over time). These services generally have guidelines for improving your reputation, but are not an outright "go/no-go" like RBLs.

5. Don't fake headers, don't lie in headers, and make sure you're including the minimum headers in messages (Date and From are required, there should be a Subject, Sender, Reply-To, and To/Cc/Bcc [as applicable]). This is one of my biggest pet-peves with valid newsletters I want to receive ending up in Junk because they fake an Outlook Express header, leave out the date, or something similar.

Optionally you should consider setting up SPF, DKIM, and DMARC. These help with deliverability, but are not required (not by the vast majority of e-mail servers).

Answer 4

Unfortunately there are many different filtering techniques and some major mail providers won't publish what they use and/or what weights are given to various tests/filters, so knowing how to get through is difficult. Basically spam has driven ISPs and users into a situation where they sometimes make it difficult for such legitimate messages (especially bulk messages such as your newsletter) to get through. I no longer consider email to be the half-way-reliable transport method it once was.

To be a little less negative and more helpful... As you are having specific problems with a particular client there may be things the program can tell you. I don't know specifically about outlook as I don't use it anywhere myself, but many mail filters inject headers into messages to list what filters were used, what the result was, and what the weighting given to that filter was. So if you look at the full source of the messages they did get moved to junk folders you may find useful clues. As an example, SpamAssassin based filters inject headers of the following form:

X-Spam-Flag: YES
X-Spam-Score: 13.371
X-Spam-Level: *************
X-Spam-Status: Yes, score=13.371 tagged_above=-10 required=5.4
    tests=[BAYES_99=3.5, FB_GET_MEDS=0.803, RCVD_IN_SORBS_WEB=0.619,
    RCVD_IN_XBL=3.033, RDNS_NONE=0.1, URIBL_AB_SURBL=1.86,
    URIBL_BLACK=1.955, URIBL_JP_SURBL=1.501]

(that example has been plucked from a genuine spam message in my junk pile)

This is not definite though as bayesian filtering and other methods that involve user training are common - so what your filters pass and fail may differ markedly to other people's even though the client was configured identically out-of-the-box. You might have to consider some other outlet for your news (many people are trying to use social networking protocols for this, with varying degrees of success).

Answer 5

Like others said, you want to avoid "looking" like a spam message when sending the email but you can't necessarily tell what will or won't make you look like spam because techniques vary.

One thing you might want to consider is sending a plain text email to your customers for each newsletter that actually contains a quick description/greeting followed by a "click here to view our latest newsletter!" message; that way you can host your message on a web server, you're reducing the size of emails (and load on your mail server) and as a bonus you can check the logs on your web server to get feedback on just how many customers are actually reading your messages vs. deleting them.

Answer 6

Detailed solution to avoid emails to be identified as spam and/or not arriving to receivers

Example situation: You have a server running a PHP website for example.com that needs to send emails. And you notice that your emails are not always delivered. (Big problem if you're a shop owner, and the customers don't receive the emails after a purchase!).

If you follow all the following steps, it should solve 99,9% of the problems. (I first thought it was possible to do only a few of them, and skip DKIM for example, but finally all of them were required to solve all the problems I had).

1. First of all, who is sending emails?

   When your PHP code sends emails, it's often with the famous PHP function mail(...). But what does this function do, under the hood? Let's run a test.php page containing <?php echo ini_get('sendmail_path'); ?>. You will get for example: /usr/sbin/sendmail -t -i. Good news, now we know which program really handles the emails!
   Now a tricky info: the name sendmail can be various programs. Even if you see sendmail in the previous step, you might have sendmail or postfix or exim, or qmail, etc. installed. Let's do dpkg -S /usr/sbin/sendmail. The answer is postfix: /usr/sbin/sendmail, ok this means we have postfix installed.

2. Look in the log file /var/mail/www-data to know which emails have not been correctly sent, and why. This might be useful for next steps.

3. As mentioned on Jeff Atwood's blog, it's time to look at reverse PTR records. (More details to be added here).

4. Add the following line in the postfix config file /etc/postfix/main.cf file:

   inet_protocols=ipv4
   

   Then restart postfix with service restart postfix. Why? Because I had problems like this when the recipient is gmail:

   Our system has detected that this message does 550-5.7.1 not meet IPv6 sending guidelines regarding PTR records and 550-5.7.1 authentication. Please review 550-5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more 550 5.7.1 information.

   The easiest solution was then to switch postfix to ipv4 only, thus this step 4 (which could be unnecessary for you?).

5. SPF DNS records. In order to prove that you are allowed to send emails from @example.com, you can add a SPF record in the DNS records of the domain example.com. I found somewhere that The DNS record type 99 (SPF) has been deprecated, so we use a TXT record instead. Let's add this as a TXT DNS record ^{(see also note 1)}:

   v=spf1 a mx include:_spf.google.com include:sendgrid.net ~all
   

   Why these includes? Because my server won't be the only one to send emails from @example.com! I configured Gmail to Send mail as [email protected] (see screenshot here), using the trusted SMTP provider Sendgrid. If I don't add these include:, Gmail wouldn't be allowed to send email from @example.com.

6. DKIM digital signature. As mentioned here, DKIM's goal is to ensure that mail content hasn't tampered during transmission. Here is the installation process in Ubuntu (useful guide here too):

   • apt-get install opendkim opendkim-tools

   • Create the keys (you can also generate keys and the relevant DNS TXT record with http://dkimcore.org/tools/):

     mkdir /etc/opendkim
     cd /etc/opendkim
     opendkim-genkey -t -s mail -d example.com
     

   • Let's put this in /etc/opendkim.conf:

     Syslog                 yes
     Domain                 *
     KeyFile                /etc/opendkim/mail.private
     Selector               mail
     AutoRestart            yes
     Background             yes
     Canonicalization       relaxed/relaxed
     DNSTimeout             5
     Mode                   sv
     SubDomains             no
     

     this in /etc/default/opendkim:

     SOCKET="inet:8891@localhost" # Ubuntu default - listen on loopback on port 8891
     

     and finally add this at the end of the postfix configuration file /etc/postfix/main.cf:

     # DKIM
     milter_default_action = accept
     milter_protocol = 2
     smtpd_milters = inet:localhost:8891
     non_smtpd_milters = inet:localhost:8891
     

   • Now let's add the public key (found in /etc/opendkim/mail.txt) to the DNS records of your domain:

     mail._domainkey.example.com. IN TXT "v=DKIM1; k=rsa; p=OqYHd...waPaQAX"
     

     Here is how it looks like with my registrar Namelynx:

     

   • Last step for DKIM: restart the mail services with service restart opendkim ; service restart postfix.

7. Test if everything works. The easiest method is to send an email via PHP to [email protected] (this very useful tool is made available by Port25 Solutions):

   $emailfrom = "Example <[email protected]>";
   $headers  = "MIME-Version: 1.0 \n";
   $headers .= "Content-Transfer-Encoding: 8bit \n";
   $headers .= "Content-type: text/plain; charset=utf-8\n";
   $headers .= "Reply-To: " . $emailfrom . "\n";
   $headers .= "From: " . $emailfrom . "\n";
   $headers .= "Bcc: [email protected]\n";
   mail("[email protected]", "Hello", "Hello!", $headers);
   

   Then see the answer of this tool, it should look like this:

   ==========================================================
   Summary of Results
   ==========================================================
   SPF check:          pass
   DKIM check:         pass
   SpamAssassin check: ham
   

   The service mail-tester.com is useful as well.

8. (Optional) Try postmaster.google.com. I used it but I don't remember if it helped or not.

9. If it still doesn't work, a solution could be to outsource email with a professional solution, to avoid days and nights of (unsuccessful) debugging. Here is a good article about this. Here is a quote: "Sending emails from your app can s***. Half the time, messages that get sent from your own server just get dumped into the recipient's junk folder." that I sadly discovered true, after weeks of tweaking.

---

Additional notes:

(1)

-all : Fail: All mail servers not listed in the SPF record are explicitly not authorized to send mail using the sender’s domain.
~all : Soft Fail: All mail servers not listed in the SPF record are not authorized to send mail using the sender’s domain, but the owner of the domain is unwilling to make a strong assertion to that effect.
?all : Neutral: The domain controller cannot or does not want to assert whether or not all mail servers not listed in the SPF record are authorized to send mail using the sender’s domain.
+all : Pass: All mail servers are authorized to send mail on behalf of the sender’s domain.

Answer 7

My online business was having problems with order confirmation emails going to spam or not even being delivered (shunned via mail servers). These were simple "here's a summary of your order" emails with one link to our site's domain. We ended up buying a few Google Apps accounts for my business. You can setup one of them to act as your SMTP server. Having Google as our mail sender stopped all of these problems.

As far as email newsletters, definitely use a service that handles opt-in/-out for you. Using anyone other than a service to send bulk mail will probably get you banned.

Answer 8

There's a new guide that's been published on email Inboxing

The most comprehensive I ever seen. A checklist of 43 different points that cover every faucet of how to avoid being marked as spam. That's continuously updated.

From setting up DNS, Configuring Authentication, Setting up reputation monitoring, Reducing your bounce rate, Testing Your Email Content, etc.

Heads to tails coverage of everything by ZeroBounce.NET

https://www.zerobounce.net/email-deliverability/

There is a new platform as well for thoroughly testing and optimizing the content of email that's pretty amazing.

Campaign Cleaner's Email Content Optimization