The world of business marketing has two absolute truths. First, a company can only go as far as their marketing efforts take them. Second, it is an ever changing field that requires constant reevaluation and testing. Many companies have employed email marketing to further themselves and make potential customers aware of new developments or sales. However, how do these companies know what is effective within the emails they send out? The answer is A/B testing. This is the practice of designing a marketing email and then creating a copy of that email with one noticeable change in place before sending out those emails to two separate groups in order to determine which version of the email is more effective. Ideally, the responses and information received from these emails should help a company design more effective marketing emails moving forward. Anyone can send an email, but not every email is an effective marketing tool. Matt Blumberg, the founder and CEO of Bolster, summed this up, “Reaching the inbox isn’t your goal. Engaging people is.”
There are numerous ways to approach A/B testing your email marketing. We connected with ten different business experts to better understand how they go about this process.
Identify your elements
Jeff Goodwin is the Sr. Director, Performance Marketing & E-Commerce at Orgain, a clean nutrition company providing products with real, organic, and quality ingredients. He believes one of the best practices when reviewing your email is to break it down into categories in order to see what can be tested.
“A high quality marketing email is made up of many elements, each of which can be changed to see how they’re interpreted by the recipient. This way, you can easily understand what is proving compelling. For example, subject lines, opening headers, image types, interactive elements, and sender titles are all elements which can be changed for testing and will likely yield different results when alternate versions are tested against each other. If you’re changing multiple elements simultaneously, it can be very difficult to understand any data received as you can’t identify what resonated with a recipient.”
Look for relevant data
Diamond Mansion is an online retailer of online jewelry which offers their product at revolutionary prices while prioritizing the customer experience. Their CEO and founder, Omid Semino, considers the correct interpretation and understanding of the data received from sent emails is incredibly imperative.
“When testing your emails, it can feel like shooting in the dark if you don’t know what it is that you’re looking for. One very important stat when it comes to testing subject lines is the open rate. This tracks what percentage of people open the email they received and will determine which subject line works better. On the other hand, click rate tracks how many people clicked on a link within the email and will inform you on which ad or link works better. If you’re looking at the wrong stats when testing it can drastically affect an email’s performance and actively work against your company.”
Sender name
The first thing people see when they receive an email is who sent it to them. This can often determine whether or not they open said email. Phillip Akhzar is the CEO of Arka, a company which provides customized product packaging for global clients. He has found testing and changing the sending name of the email to be a successful method.
“If you view your email inbox on your phone or computer, your eyes generally jump towards the list of names or companies which have made their way into your inbox. Personal and work emails take priority followed by anything else, like advertisements. If you can design your sender name in an enticing manner you’ll likely see the open rate of your emails increase. Let’s say your company name is Evergreen Clothing. You could simply list your sender name as the company name, or you could try to get creative with it by listing it as “Mary at Evergreen Clothing” or “Evergreen Clothing Discounts”.
Sample size
Jeremy Gardner is the CEO of MadeMan, a health and beauty company which specializes in skincare products for men. He believes that any testing should be done with appropriately sized sample groups in order to achieve accurate results.
“If you send too many emails to those on your company email list, it’s a quick way to turn them away from your company or make them unsubscribe. This is why, when testing, you should only email a portion of your total list. Most companies operate their testing group with twenty percent of their list. So, ten percent receives email A while the remaining receives email B. Moving forward, the outstanding eighty percent should receive whichever email performed better in the testing process. It makes zero sense to test across the entire email list.”
Subject line and preview
Straight Up Growth is a marketing agency which aims to help their clients grow through the use of Amazon’s search engine. Their co-founder, Daniel Tejeda, considers testing of both the preview text and subject line of an email to be critical.
“Most people decide if they’re going to open an email at first glance so first impressions are everything in email marketing. One word or a different phrase summarizing the email can make all the difference in the email open rate regardless of the body of content. Play around with both the subject line and the preview text of an email to see which is received better. The idea is to stand out within someone’s inbox.
Timing
Some people have push notifications enabled and they will see incoming emails on their device as they come in. However, others check their inboxes periodically and emails further down the list may be overlooked. In this case, timing is everything. Chris Bridges, the CEO of VITAL, a rewards based credit card, understands this.
“Often, people are quick to check their emails earlier in the day and as the day wears on, it becomes less of a priority. Because of this, it is imperative to send emails earlier in the day. On top of this, you can also track how long after an email was received it was opened. Knowing this should help your marketing team understand how appealing your efforts are.”
Personalization
Chris Gadek is the head of growth for AdQuick, an out-of-home advertising technology company. He has found testing personalization within marketing emails to be an effective way to see what works and what does not.
“Technology makes it very easy to add a person’s name to the introduction of an email. Doing this can result in more opened emails depending on how it is phrased. However, some recipients may feel their privacy is being violated with this practice if the phrasing is off. This can result in lost subscriptions or blocks. For this reason, it’s important to test personalization in a variety of ways to ensure that your message feels engaging and welcoming, not off-putting.”
Images
Triple Whale is an all-in-one hub for businesses to monitor and operate data and applications that they use for ecommerce and marketing. Their CMO, Rabah Rahil, suggests trying different images and layouts for emails.
“A beautifully designed graphic does a much better job of capturing a reader and getting the point across. That’s why we see so many more emails built entirely around or solely featuring images. But, some images are better than others. Fonts, layouts, product placement, colors, and information are all variables here that should be played with and tested to see how effective they are. Don’t be afraid to get creative in this area but remember that the viewer of the image knows far less about the content than the creator. Keep the message concise.”
Look for significance
In these types of tests, numbers are the deciding factor in eliminating an element or continuing to use a practice. However, the numbers should show a clear way forward before a decision is made. WTFast is a network solution company which aims to combat internet connection issues for video games. Their CEO, Rob Bartlett, believes this.
“Not every test is designed perfectly and you should take this into account when evaluating the results from any A/B test. Sometimes, your opposing emails will yield similar response numbers. Other times, there will be a clear landslide towards one email. If there isn’t a clear winner, then more testing should be done before finalizing any elements of an email.”
Software
Joshua Chin is the CEO of Chronos Agency, a company boasting a lifecycle marketing agency for ecommerce companies. He considers the right software to be vastly important to managing the entire process of A/B testing.
“Any company can manually create email lists, send their test samples, and create spreadsheets of the returned data before interpreting it. But doesn’t this sound like a lot of work? There are many marketing email software solutions to streamline this process from top to bottom. Not only will they do all these things for you, the data interpretations will be much deeper than the old school approach.”
A/B testing an email marketing campaign is no small task and requires an ample amount of attention. When done right, it can greatly impact the popularity and sales of a company. Email marketing cannot be solved in an afternoon and everyone involved should continue to improve the campaign based on the results of the test. Jordie van Rijn, an independent marketing consultant, put it best, “Design like you are absolutely right, then optimize like you were wrong from the start.”
