Read Time: 18 min

Delivering Episode 21: Measuring Emails With Litmus

In this episode of Delivering, host Jason Rodriguez digs into how email tracking works, what can and can’t be tracked in an email campaign, and how marketers can put Litmus to work to gain insights into their email marketing efforts to improve future emails.

Delivering is brought to you by Litmus. Litmus is the only platform that helps you send email with confidence, every time. Over 600,000 marketing professionals use Litmus’ tools to build, test, and analyze better email campaigns faster. Head over to Litmus.com to start your free 7-day trial of Litmus, and start sending better emails today.

Episode Transcript

Welcome to Delivering, a podcast about the email industry, from strategy to design, code to leadership, and everything in between. I’m your host, Jason Rodriguez. Delivering is brought to you by Litmus—the only platform trusted by professionals to help you send email with confidence, every time. Over 600,000 marketing professionals use Litmus’ tools to build, test, and analyze better email campaigns faster.

Head over to litmus.com to start your free 7-day trial of Litmus, and start sending better emails today.

Be sure to subscribe to Delivering on iTunes or Spotify to listen to future episodes and join the conversation on Twitter using the hashtag #DeliveringPodcast.

One of the most important parts of email marketing is measuring the impact of your campaigns.

It can be tempting to focus all of your attention on the creative production of emails—writing copy, designing graphics, and coding the world’s greatest responsive template—but most of that work goes to waste if you can’t prove that it actually makes a difference to your subscribers. And the only way to prove that is by paying attention to your email metrics.

Properly measuring your emails, and routinely reviewing key metrics, will let you know what your subscribers care about, how they’re interacting with you campaigns, and give you what you need to improve future campaigns. Not only to achieve your business goals, but to provide more value for your subscribers—which is what we’re all here for in the first place.

Measuring email is so important, it makes up two of our three pillars of a successful email program: post and perform.

Unfortunately, email tracking and measurement is still relatively misunderstood, even by people with a thorough understanding of technology or who have spent some time in the industry. And it’s not without its controversy, too.

You can listen to Episode 16 for some of my thoughts on the ethics around tracking emails but, for now, let’s assume that you do need to track and measure your email campaigns. If that’s the case, then you’re probably left with a few questions like:

  • How does that tracking work?
  • What can be tracked in an email?
  • What can’t?
  • Which metrics should I actually pay attention to for my email campaigns?
  • And how can Litmus help me track my campaigns more effectively?

In this episode of Delivering, we’ll look at answering each of those questions.

Starting with… how does email tracking work? For that, I’m turning to one of the pioneers in email tracking—our very own Brendan Caffrey. Brendan is our Vice President of Engineering and Chief Architect at Litmus, and has been instrumental in enabling email marketers to better track their campaigns with Litmus.

The kind of first thing that we have to understand when we’re talking about emails, you have very limited to access to things, right? We just really have, you know, HTML and images. There’s no scripting. There’s no, there’s no real code executing. I want to be careful with that. People get a little sensitive about the HTML code or not, but there’s no, you know, there’s no real advanced logic being executed when you’re looking at an email.

So, all email tracking is basically of the images today, right? So, every email tracking service, whether it’s Litmus or an ESP, they’re just  going to put an image, or something that appears to be an image, you’ll put an image tag in an email. And then, just on the other side of that, instead of having like a static image like you would normally do, you just have a web server that’s, you know, serving requests and listening effectively.

And so, if we listened, was like, “Oh, okay. I know that, Jason recorded this open because his image was opened or viewed”  and the first thing a lot of people point out is, is the obvious thing  this is obviously wildly inaccurate. I have no idea if you actually opened that email. I just know that that image was loaded. That’s  literally the only thing that we infer that you probably opened that message. Lots of things could cause that to open, you know, well, we know, for example, people like Gmail, they’ll use their image cache to kind of prefetch images and things like that. So there are a lot of scenarios in Gmail where it could look like an open when it’s not.

We know that there are some, though fewer than there used to be, spam appliances that’ll like scan content before it gets to a user and their messages. But, that’s pretty rare these days. So yeah, I mean, it’s at the most basic level, it is just tracking that someone has loaded it.

Yeah. So because we do just, we just have that one request. So someone’s hitting our server and they’re asking for an image. Right? And so in that, you know, couple of seconds we have, then there’s very little that we can track.

One thing that we can track, though, is which email clients subscribers are using to open emails. This is vital data to email designers and developers that need to know which clients and rendering engines they need to code for. As we all know, no two email clients are created equal, and each displays our emails differently depending on what HTML and CSS code is supported.

Anything that kind of loads HTML or serves a request, and in part of that request, they include something called a user agent.

The user agent is a thing that says, “Oh, this is the type of client that is loading your HTML resource.” And so, you know, super common is that people will see, as you know, well, there was Firefox or Chrome or Internet Explorer, Safari, right? But that’s loading their webpage. So you use Google Analytics for these different things and you see this little pie chart of who’s using  your website, right?

If they’re just parsing those user agents, similarly, email clients will pass, most clients will pass, a very specific user agent based on what version it is. iPhone and Android will pass these user agents. The reason these are somewhat tricky is that they’re not, you know, it is up to the person making the email client to decide what to put in that user.

Frequently it can be inaccurate, A really good example of that is that we know that Microsoft stopped changing their user agent after Outlook 2016, so anything from 2016 and on just appears like Outlook 2016, they’ve not changed these agents since then. So it’s impossible for us to tell because Microsoft is still serving this old user agent.

Beyond that, there are a few other things that you get. So when people make a web request, you have something called a referrer, which is like the website that referred you to this, to this website, right? And so, for web mails and the ones that don’t do image caching, the referrer will be, you know, the site that’s hosting the kind of mail provider. So, you know, well it can’t AOL is an example anymore because they do image caching, but a lot of the foreign web mails, Comcasts, and things like that, you’ll be able to tell where the request is coming from. And so that’s user agents, and referrers. And then the last thing that we do, pretty rare that we do it, but we know  that certain clients only detect or support certain HTML elements or certain CSS styles. And so we’ll strategically, if you look at our HTML snippet for tracking, it’s not just an image, it’s quite more complicated than that. And so we’ll use our knowledge of how email clients work and say, ah, I know that only Thunderbird loads this kind of div. So if I, if this specific image is loaded, that means it had to be loaded in Thunderbird, cause it’s the only email client. So we can do tricks like that as well.

Even with the knowledge of how email clients identify themselves, email tracking has significant challenges.

Yeah. Probably the biggest one that most people are aware of is the image caching that is now used by like Gmail and Yahoo and some of the larger providers, you know, it’s a feature for their users, right? Because i ages load faster, there’s that element of privacy in that like, I can’t, I can’t detect where you are in the world.

But it is a little, obviously, limiting for email developers because you know, specifically on the email side, you don’t know who’s using the Android Gmail app versus the web app versus iOS. And those things would be helpful to another developer to know, you know, how do I make my email responsive? No one’s using the the web client to really focus on mobile or vice versa, and you just don’t have that information or that’s probably the biggest limitation out there, especially how big email is just market-share wise.

With all of those challenges, I was eager to hear Brendan’s thoughts on the future of measuring emails.

There’s going to be more interesting things is one thing. If you think about what the advent of AMP and stuff like that potentially, there’s more things we could do there to detect, you know, we can put a tracking code specifically on the AMP side that maybe could, could have slightly more capabilities.

There is also a bit of a movement, because of how poorly implemented user agents are, to kind of deprecate those in favor of a kind of more advanced standard, which would definitely open things, potentially open things up for us. And email tracking is one of those things that is just constantly evolving and there’s always new clients coming out. There’s always new things for us to investigate. But it’s a, it’s a fine line of, you know, how much effort we want to go to, how many hoops we want to jump through and like, where the line of like, Oh, this is good enough. Right?

I mean, it really depends on what your aim is, right? For the most part, folks’ aim with tracking is to know how their audience is segmented, to know how their messages are performing. And so to those two goals, like what we have right now is probably good enough. But, you know, people always want more information, right?

Brendan makes a great point. Our goal as email marketers is to understand a few things:

  1. Where our subscribers are opening so that we can create a good experience for them in those email clients.
  2. What kind of content is resonating with subscribers.
  3. How we can use that knowledge to improve future campaigns and our wider marketing strategy.

Even with the current limitations around email tracking, we largely have the tools needed to answer those questions.

One of the ways of measuring emails I didn’t discuss with Brendan is link tracking. Email marketers, and the tools we use, have a few different ways to track which links are clicked in an email campaign.

Most ESPs will use some sort of link rewriting to gain insights into how content is performing. It’s kind of like how Gmail caches images on their own servers, but with links. ESPs will rewrite whatever links you put into an email and route traffic through their own servers to log information on those links. That’s why, instead of seeing a link like litmus.com/blog, you’ll see something like litmus.cmail19.com, which is Campaign Monitor routing subscribers through their servers to track clicks before passing them onto our servers where the content is hosted.

Even if an ESP isn’t rewriting links, most traffic from an email is pointing to apps or websites which have tracking scripts enabled. Website tracking (which has its own ethical pros and cons) can give email marketers insight into subscriber and customer behavior.

If you’re using website or app tracking, then it’s important to include tracking parameters in your links to properly attribute traffic and see how your campaigns and content are performing.

Tracking parameters are all those bits of text you see added onto the end of links, usually following a question mark symbol. You’ve probably seen stuff like, “utm_source=email”, or similar. These allow tracking services like Google Analytics to collect additional information from links and provide more granular reports on where users come from and how they flow through your content once you’re on a site or in an app.

Fun fact: UTM actually means Urchin Traffic Monitor and comes from the service Urchin Tracker, which was an early pioneer of web tracking. Urchin Tracker was acquired by Google in 2005 and formed the basis of what is now Google Analytics. UTM just stuck around for the ride.

Litmus even has a built in UTM Manager that allows you to quickly add UTM codes to links in your email. Next time you’re in Litmus Builder, click on the “Tracking” menu option and then the “UTM Manager” option. You’ll be able to add and save your UTM tracking code and then either add it to every link in an email or select which links you want to be tracked. It’s a massively useful tool and saves you the time normally spent copying and pasting that code on all of your links. It’s actually one of my all-time favorite features in Builder and one of those hidden gems that everyone should probably be using.

OK, so back to link tracking…

Link tracking gives email marketers all of the insights they need for seeing what content subscribers actually care about. If you have a call-to-action being tracked, you can see how many subscribers actually clicked on that CTA. And, if you’re using UTM codes, you can better follow that subscriber through your website and your funnel to see their journey through your website or product.

Combined with open rates, which—as Brendan mentioned—are somewhat unreliable—they can give marketers a better idea of what content actually works.

There are a few variations on both open and click rates. Everything from open rates to mobile open rates, click-through-rates to click-to-open ratios, and conversions. Each has their pros and cons and can tell email marketers different things about their subscribers and campaigns.

But that’s going to be the subject of another episode.

What’s important for email marketers to understand right now is that there are a few different ways to measure email. And no single way should be considered the sole source of truth. With various limitations around email tracking, you need to take all of those methods into account to learn what they mean for your campaigns and your marketing program.

That’s largely why you should use multiple tools for measuring emails. Email service providers are good at giving you a look at open rates and clicks but a tool like Litmus’ Email Analytics will provide insights into different subscriber behavior.

Using Litmus, you can see more detailed metrics on which email clients, platforms, and rendering engines your subscribers use, to geolocation, forwards and prints, and even how long subscribers are keeping your emails open. And, we recently introduced aggregated reporting in Litmus, which allows you to compare performance across past campaigns to get a better look at subscriber trends.

One other thing we’re doing at Litmus beyond investing in Email Analytics and all of our research and content around analyzing campaigns post-send to improve future performance, is embracing the theme of measuring email in June for Litmus Live.

If you haven’t heard yet, go back to the last episode of Delivering to see how we’re bringing Litmus Live—our annual conference for email professionals—online in the light of the global pandemic and our commitment to keeping our community safe. Each month, we’re looking at a new key theme that email marketers need to thoroughly understand to keep current and stay competitive. In June, we’re all about “measuring emails and telling stories with data”.

In our workshops, we’ll be looking at how email marketers can get a better handle on—and understanding of—their email metrics. When it comes to telling stories with data, I’ll be leading designers and developers through how to put that data to use in interactive emails with HTML, CSS, and AMP.

On June 16th, we’re hosting our second Litmus Live Day. Anne Tomlin, from Emails Y’all, will be leading our practitioner session with advice on how to build templates with dynamic content and personalization in mind. And, in our marketing leadership session, we’ll be having a fireside (or at least screen side) chat with a special guest on how to measure and understand your subscribers better.

If you haven’t already, head over to litmus.com/conference to sign up for Litmus Live. The monthly Litmus Live Day sessions—and our weeklong Litmus Live Week in September—are completely free. Sign up once and you’ll be able to hop on every month this year to learn from some of the best minds in email, right from home.

That wraps things up for this episode of Delivering. Head over to litmus.com to start your free 7-day trial of Litmus, and start exploring all of our tools that make measuring and improving your campaigns easier than ever before.

And be sure to subscribe to Delivering on iTunes or Spotify to listen to future episodes and join the conversation on Twitter using the hashtag #DeliveringPodcast.