The Results Are In!

by | Dec 23, 2020

If you read my last blog, you’ll hopefully know that I was going to conduct a split test to find out whether sending from my old domain was still getting better results than sending from the Deliverability Dashboard domain.


The Results Are In… But Do They Mean Anything?

Well… maybe, maybe not. I’ll let you be the judge. 

My audience wasn’t big enough to gain too many insights from the result. But there was a clear overall result.

The Good News

First things first. I saw a much better performance from the new domain compared to the old domain.

This was a pleasant surprise.

Because, until recently, I’ve consistently seen poorer open rates and higher anecdotal reports of emails going to spam with the new domain.

I’ll never know for sure, but it’s possible that sending a couple of broadcasts from the new domain might have been enough to “warm it up” so that I’d see good performance anyway.

Anything Else to Report?

I’m going to report the results in more detail below, and would love to hear what other conclusions you come to after looking at the numbers.

Here are my thoughts:

  • The overall open rate was a little higher than usual… 
  • Google still generally gets much better open rates than Microsoft
  • The old domain only performed better on the segment of engaged non-Google, non-Microsoft contacts (16.7% of the overall audience), but the new domain performed better in every other case
  • Microsoft open rates are significantly lower for less-engaged contacts (which corroborates the theory that Microsoft is more likely to route emails to the junk folder sooner than Google as contacts become more and more unengaged)

Here’s the Detail

Firstly, here are the detailed results:

Sent From Sent To Segment Sent Opened Open Rate
New Domain Google Engaged 382 230 60.2%
Old Domain Google Engaged 381 189 49.6%
New Domain Microsoft Engaged 48 25 52.1%
Old Domain Microsoft Engaged 47 18 38.3%
New Domain Others Engaged 124 62 50.0%
Old Domain Others Engaged 123 77 62.6%
New Domain Google Unengaged 97 30 30.9%
Old Domain Google Unengaged 99 22 22.2%
New Domain Microsoft Unengaged 38 3 7.9%
Old Domain Microsoft Unengaged 34 1 2.9%
New Domain Others Unengaged 57 8 14.0%
Old Domain Others Unengaged 52 6 11.5%

And here’s a summary:

  Sent         Opened    Open Rate
All Engaged 1105 601 54.4%
All Unengaged 377 70 18.6%
Engaged New Domain 554 317 57.2%
Unengaged New Domain 192 41 21.4%
All New Domain 746 358 48.0%
Engaged Old Domain 551 284 51.5%
Unengaged Old Domain 185 29 15.7%
All Old Domain 736 313 42.5%

How Did I Warm Up the Domain? 

Firstly, I started by sending transactional emails from this domain. In this case, it was the email containing login details, sent every time someone registers for the Email Health Check. Unsurprisingly, people look out for this email and open it. Which was a good start because I immediately got high open rates, albeit for a low number of contacts.

Secondly, I started sending emails from the new domain to my most engaged contacts. I changed the email address, but not the “Adrian Savage” name that’s shown next to the email address. Again, because they had engaged with me recently, and saw a name they recognised, they were more likely to open these emails and again, I got high open rates.

Then, I switched everything over for one email broadcast and sent to all of my contacts who’d opened something in the last 90 days. That was when I saw a lower open rate, but last week’s split test seems to have confirmed that things are good enough after all. 

So, to sum up, as long you manage your engagement carefully, you can migrate from one domain to another and still get good engagement fairly quickly… as long as your previous domain has a good enough reputation.

And if you use Deliverability Defender to manage your engagement and use the Email Health Check regularly to monitor your health score, you should already have that good sending reputation.