27 Jun 7 steps for testing your newsletter

7 steps for testing your newsletter

We all have a website or newsletter, but do they bring the desired effect? Does your sales increase after the launch of a newsletter? Or do you notice an influx of people on your website? One way of measuring this is by using the A/B testing technique which shows you how your target audience is reacting to your messaging.

You might have heard about A/B testing before, but what is it exactly? I am sceptical when it comes to using this technique. Why? I’ll explain in this blog. Yet it’s a technique that’s often used when testing a landing page on a website or newsletter. Read on to find out more about what it is and how it works.

Are we going for green or red?

When you’re curious to know which colours or imagery works best with your target audience then A/B testing is a handy tool. What you do is the following: you show two versions of the same message to your target audience. You split your audience into two groups, show version A to group 1, and the other half receives version B – hence the term A/B testing. For example, A has a red button, and B has a green one. Or you might prefer to test other elements like the layout or images. Once you’ve collected the data, you’ll see which version works best for you to incorporate into future versions of what you’re testing.

And that’s why I’m sceptical…

Because I’m one of those people who says, ‘To really know, you have to send everyone the same two versions, that’s the only way to be sure of knowing which version has the best effect.’ Why do I say this? Because how do you know what your target audience likes? It’s a gamble. I use myself as an example, if you send me version A and I don’t click on anything, who is to say it has to do with the red button? You need to send me version B as well to see if I would click on the green button instead. It could have nothing to do with your button colour but with your imagery or messaging that doesn’t appeal to me. So, who knows how anyone would really react? To me, it makes absolutely no sense. As you could have lost a conversion by not showing me the second version. This caused lots of discussion between me and the online team, the conclusion being they didn’t have an answer to justify that I was incorrect.

But don’t let that stop you from doing it. And to help you with that, I have made a step-by-step guide for you to apply yourself in practice.

1. Decide what you’re going to test – newsletter, landing page, advert, etc.

To test you first need a ‘product’ to test, for example, a newsletter. Then decide what you want to test – a variable. That can be an image, buttons, or text. Stick to one variable per test. When you test several things at the same time, you can’t really see what works and what doesn’t.

If you want to test multiple variables, then go for multi-variation testing. This way you test several variables at the same time as well as the combination of these variables together – this saves you several A/B tests but a technique I won’t cover in this blog.

To illustrate, I’m using a newsletter where I test the buttons.

2. What’s the purpose of your test

Testing only makes sense when you set a goal. For example: ‘Which newsletter leads to more conversion, the red button or the green one?’ Version A is the original and therefore has the red button. But as we want to test the effect of the button colour, version B gets a green one.

3. Who will see which version

Next, look at your mailing list – this is your target audience. You will then divide these into two groups, as you need two groups to test. To conduct a good analysis, it’s desirable to have between 25-100 people in each group or more. And make sure that they’re equally distributed in terms of numbers, so not 25 in group 1 and 75 in group 2. An equal number ensures a realistic picture.

4. And the winner is – numbers

After sending out the two newsletters, you’ll see how your target group reacts – the analytics. By determining a condition in advance, for example, ‘The green button must yield 15% more conversion than red.’ The data will show which version has the best conversion. So, when green has done better than red, you can decide to use the green button in future newsletters. There’s also the possibility that there’s no difference in whether red works better than green. When this is the case, it will only confirm you were already on the right track using red and that you don’t need to adjust the colour of the button. In that case, choose another variable to test in your next newsletter.

5. Duration of testing

An A/B test has no fixed duration and differs not only per ‘product’ that you test but also per company. It’s important to collect enough data to be able to assess the test properly. It may be the case that when testing a landing page, you take the duration of the campaign as your test time frame. Without enough data, you can’t tell whether version A is better than version B.

6. What does your target audience think – feedback

To me, this is a strange one as you’ve shown your target audience only one version. So how can they give feedback without having a comparison? One way would be to send out a survey to both groups at in later stage where you show both versions of the newsletter and ask specific questions such as ‘Why did you choose the red or green button’. But when you send out a survey without showing the two versions, it’s very difficult to answer these specific questions. I propose to set up a test panel of followers/customers and send them a targeted email with questions and examples. If necessary, give them a reward (10% discount on your next order) for completing the survey.

7. And what to do with the results

After receiving all the data evaluate and analyse it. Carefully look at the findings and draw conclusions from the data. If your data clearly tells you that the green button created more conversions compared to the red one, don’t be stubborn and keep using the red button. There’s a reason why your followers chose the green button. Or you might as well not have tested.

One note though, do take into consideration the colours used in your corporate identity and stay close to those. You don’t want to have a green website and then use a blue button which makes the aesthetics look really bad and unprofessional.

How to proceed

Okay, so now you think, ‘Great stuff! But I have no idea how to do this myself!’ Then book the Style Scan and I’ll gladly take you through the process or can by taking it off your hands, so you can enjoy doing what you love doing most.