A/B Testing Best Practices
Chris Mulvaney is the CEO of CMDS. I make things... I’m the creative entrepreneur with passion for (re)making brands and inventing solutions to problems no one knows exist.
Determining the best marketing method to use can be tricky. Simply choosing a method based on intuition, or a personal feeling isn’t always (or ever) the best solution. With so many new marketing tools and forever-changing third-party algorithms and statistics, a lot of planning and testing is needed to determine best practices. To help, we’ve outlined A/B Testing best practices.
What is A/B Testing
When relating to marketing, A/B testing is comparing two (A and B), controlled marketing methods with a specific goal to determine which tactic performed or delivered the best results. A/B testing, also known as split testing, can be used in countless ways: email marketing or ad campaigns, creating landing pages, or testing call-to-action buttons with goals such as conversion rates, clicks, form completions or even purchases, to name a few. With A/B testing, you are essentially observing how one test performs over the other, or how they perform alongside each other. The method that delivers the best conversion rate optimization (CRO) is considered the best practice.
There is one important thing to note about an A/B test. It is essentially only testing two variations – A and B. Although A/B testing is also referred to as split testing, the definitions are different. A split test is testing a variation of more than two elements. Whereas A/B testing is comparing two methods. An example of an A/B test would be having two banner ads with varying copy. A split test would be having 4-5 banners with the same copy, but slightly different graphics or layout. An A/B test is only when one element is changed.
Conversion rate optimization testing adds incredible value to marketing efforts. The better the CRO, the better chance of your marketing performing to its full potential and ultimately reaching the overall goal(s). In order for an A/B test to be effective and provide reliable results, A/B testing best practices should be followed.
A/B Testing Best Practices
Step One: Determine the Goal
Although A/B testing is considered a simple method, careful planning is required. Best practices dictate that your first step should be to determine your goal. Goals for your campaign will vary depending upon medium, whether you are B2B or B2C, and type of business.
Without first establishing a goal, the results of the test will be irrelevant. These goals can be business goals, such as phone calls or inquiries, or website goals, such as site visits or clicks. Some examples of specific goals are:
- Conversions
- Clicks
- Impressions
- Form completions
- Revenue
- Downloads
Step Two: Determine Location of Test
The second step in A/B testing best practices is to determine to test. Let’s take a website for example. Before you can launch your test, you need to decide which page the A/B test should be assessing. This could be on the homepage, the check-out page, or even on a blog. In many cases, you would want to create a landing page for your A/B test.
In order to determine the best page, keep in mind your specific goal as well as your current analytics.
Things to consider:
- Reference your website’s analytics and determine which pages are under-performing and develop A/B tests to address issues
- Determine where marketing campaigns will be driving traffic and make sure that page is properly tested
- Examine site analytics to determine top pages, pages with longest time on site, pages with highest and lowest bounce rates, most common entry pages, etc.
- Whether you need a landing page for a specific campaign or goal, or whether you can use an existing page on your site.
Step Three: Select A and B Elements
At this point, you have determined the goal and the location of the test. The third step is determining what to test – A and B. Although reviewing your analytics is imperative, A/B testing can help solve the questions analytic reports do not answer.
For example, if you are an eCommerce site and your goal is to sell a certain number of products, you may want to test your call-to-action button. A few ways to test this are:
- Copy – Determining what the call to action should say – Buy Now vs. Click Here
- Placement – Determining the placement of the CTA button – in the center of the page vs in the top right
- Color – Determining the color of the CTA button – blue vs. red
- Shape – Should the CTA button be oval or rectangular
- Size – Doe it work better large or small and subtle
- Special Effects – Is it better to include a drop shadow, animation, arrows, etc.
These may sounds like simple things, but they play a huge role in your marketing initiatives. You will also want to determine the length and size of the test. For example, if you are A/B Testing an email, you may want to start with a smaller list to see significant results. However, if you are A/B testing a website, or specific web page, running the test for a significant length of time is crucial to obtain a certain number of views, or to see the desired results.
Step Four: Review Your Testing
Once your A/B testing has run for a statistically significant length of time, the next step is to review your results. The results will determine next steps, however, they will not always be concrete. Based on the results, it could require more testing, or may inspire to test a third option.
Take, for an example, the A/B test to determine which CTA button color resulted in the most clicks. You tested the colors red and blue, with red received 15% more clicks. Results like this make it apparent what the next step should be – use the red button. However, if the two had similar results, the next step could be to either 1) continue running the test to achieve more feedback or 2) decide that color may not be a factor and begin a new test using another variable such as button copy or page placement.
Recapping A/B Testing Best Practices
A/B testing is a great, data-driven method to test website design and marketing campaigns. Keeping A/B testing best practices in mind when forming your test will ensure higher results and better performance across the board. These best practices not only streamline the testing, but will also make the most accurate results. Avoid testing multiple variables at once, confusing the results with indeterminate data.
Most importantly, always keep testing! Every web page, email, digital ad, etc can be improved — but you never know how until you test!
