Email Testing and QA: Your Top Questions, Answered

Email testing and quality assurance (QA) is one of the most vital parts of your email workflow–but it can be one of the most challenging in terms of time and resources. In our August Litmus Talks webinar “Email Testing: Your Key Ingredient to Winning Campaigns,” Litmus email team members Carin Slater and Jaina Mistry recently chatted about all things email QA-testing with Ryan Franciscus, Director of Product at Acoustic. Read on for the key takeaways (and answers to many webinar attendee questions!), or watch the full webinar on-demand at your convenience here.

What does email QA testing mean?

The most basic definition of an email QA test includes making sure your email is going to show up in the inbox (and function) as you intend. That includes confirming that:

• Email templates and headers/footers are functioning
• Email copy and content is error-free
• Links in email copy, calls to action, images and buttons direct as intended
• Email content renders properly in various email clients 
• The email is accessible

If your campaign involves email personalization, contextual targeting, or dynamic content as part of a broader lifecycle marketing strategy, the email QA process also involves checking that the email is functioning properly based on who the content in a given version is intended to reach.

How can you streamline the email QA process?

The email testing and QA process is rarely a one-person show. Based on the complexity of your email campaigns and the size of your organization, it will likely involve sign off from a number of stakeholders, both in and outside of your email marketing team.

Follow these five steps to streamline your process.

1. Determine when your email QA testing process should begin

Email QA testing might seem like one of the last steps in your email workflow, but you’ll save time and reduce inefficiency if you consider it part of your process from the very start.

For example, Francisus says his team’s email QA process starts at the planning phase. Including QA before you start executing will dictate how well you’re able to resolve defects, bugs, improper styling and missing links, and can help you identify and eliminate issues or roadblocks that might come with securing approved copy and content.

2. Never assume an existing template (or header or footer) will work

Once his team determines which template they’ll use, Franciscus says they move right to testing the template using Litmus Previews. If there are issues in rendering (and changes that need to be made) they know upfront, so it can be built into the project timeline.

“Just because you tested something a day or a week ago doesn’t mean you can skip the testing process. Don’t assume that templates are all perfect, or that things are going to work as you expect,” says Franciscus.

3. Use email QA testing to increase organizational transparency

While “many cooks in the kitchen” can lead to a more complex email QA process, there are ways to turn that into an advantage. For example, email QA testing can help increase transparency and involvement around email communications that could have an impact on cross-functional teams, like go-to-market and sales.  Involving other groups can also be an opportunity to collect valuable feedback on issues like copy or images that need to be adjusted based on cultural sensitivities or language differences in global campaigns.

4. Centralize your feedback collection and approval process

An email collaboration tool like Litmus Proof empowers you to cut hours off of your review and feedback processes by moving feedback to a central location. Your stakeholders will know exactly what type of feedback you seek, where to find the email content to review, and where to leave their comments or approvals. With a centralized tool, you eliminate email threads that are hard to track.

5. Prioritize email clients when possible

There are more than 100+ email clients that your subscribers could theoretically use. If you work in a  highly regulated industry, testing each and every one might be a requirement regardless of what percentage of your audience uses it.

But if you do have some flexibility, a tool like Litmus Email Analytics can help you determine which email clients the majority of your subscribers use. In turn, you can make informed decisions to set priorities on your default email client list instead of trying to make every single email perfect, everywhere.

Tips to stay focused with QA testing

As important as email QA testing is, it can be laborious and hard to stay focused. Try these tips to make sure your QA game is on point:

Experiment with gamification

Franciscus says his team has made the QA process a little fun with an occasional “find the Easter egg” hunt. It uses badges that reward people for catching the most bugs, and helps to reinforce the importance of QA testing while making it feel less repetitive.

Give yourself breaks

As simple as it sounds, Litmus’ Email Marketing Specialist Carin Slater says that simply taking breaks at various points in the QA process and coming back to look again with a set of fresh eyes can make a big difference in what you’re able to catch or miss. “Slow down and double check, before you move on. Then..check again,” says Slater.

Know your work style and flow of your workday

Are you more focused and mentally sharp in the morning, afternoon or evening? Jaina Mistry, Litmus’ Director of Email Marketing, recommends scheduling your QA tasks based on your personal workstyle, and the flow of your workday. Scheduling your email QA at a time when you have mental clarity and little distractions can have a major impact on your ability to do a thorough job.

Ways to incorporate accessibility testing into your email QA process

Knowing how your emails will sound in a screen reader should be a standard part of your email QA testing process. Try these tips:

• Use an email design system that includes accessibility
• Use Litmus’ email testing tools to perform accessibility checks. You’ll immediately know if your email contains text alignment or justification issues, or certain HTML elements that impact the way subscribers with cognitive or visual impairments experience your emails. You can also hear an audio recording or view a transcript of your email before you hit send.
• Put your email into your ESP. This may still tell you there are accessibility issues–but if you’ve done your due diligence with the steps above, you’ll know which are accurate.

Don’t forget to QA your A/B tests

Running an A/B test can pose some additional challenges in your QA process.  “If the test is just for subject lines, I’ll test one email,”  says Slater. But for a design test, she’ll QA both of them. At the end of the day, it’s the email QA testing mantra: don’t take things for granted or assume that they’ll work properly.

When there’s personalization involved with A/B testing, QA can get even more complicated. But Franciscus points out that when you’re testing, you’re modifying code and modifying content–and that could impact other areas of the email. Use the A/B testing split and send it out prior–and consider doing an A/B test of your test list. “Take an iterative approach and be very methodical,” says Mistry.