Content Marketing Web Development

A/B-Tests: Why They Are Pointless – As Long As You Haven’t Done Your Homework

You have heard of A/B testing and would like to use it to test your website? Through targeted A/B tests you can optimize the conversion rate and other visitor numbers in the long term. But before it comes to testing, the basic requirements for a positive user experience should be fulfilled. You can find out what these are and what necessary things you need to know about A/B testing here.

Wild testing? Rather not!

The homework: already at school an irksome topic. Even without doing it, you often came quite well through the day. But as soon as the knowledge was put to the test in the class assignment, it was clear: There are so many mistakes, and a top grade is not guaranteed. Just a hope that others would do just as bad as you and you would not stand alone stupid …

In a similar way, the homework can also be assigned very well to the practice of online shops. Here, too, the homework must be done to have a chance of becoming the best in the class, that is, beating the competition.

Before you test smaller details of your shop, the big picture should be coherent: What about the user experience and usability of your website?

A/B tests: what, why, how?

Definition A/B Testing

An A/B test or split test compares two versions of a webpage, app, or newsletter to determine which version performs better, both in terms of the user experience and the conversion rate.

A/B testing can be used, for example, to check a changed web design or changes in user guidance (new positioning of the search field, changed filtering …).


How does an A/B test work?

  • First, an A/B test target is set (for example, “most Newsletter subscriptions,” “most registrations”) and the test object is produced in two versions.
  • The stream of visitors is split randomly into two groups (A and B), which display either version A (the original) or version B.
  • Afterwards, the reactions of the users are compared, for example, with which variant most of the registrations for the newsletter were made or the most frequently clicked on a link.

Benefits of A/B testing

  • The planning scope of the test is limited: You only need two variants of the page to be tested and have to set a test target.
  • A/B tests provide useful results even if you see only a few visitors per day on your site and few conversions per day.
  • The evaluation of the A/B test is simple: the variant that performs better has won.

Disadvantages of A/B tests

  • If you have few visitors and conversions on your site, it takes a long time to get significant test scores.
  • You can not test multiple changes at once. If you compare two variants of a website, you should make only one change in order to evaluate the test result meaningful and to draw meaningful conclusions from the reaction of the user.
  • A /B testing done on existing customers can be detrimental to the user experience; namely, when the users see big changes, but later reversed. This only leads to confusion. Accordingly, you should only perform your A/B test with new site visitors.

First, do the homework …

… before you dedicate yourself to the subject of A/B-Testing. As useful as the A/B test method may be in terms of conversion rate and other metrics, it is not a panacea for poorly performing sites. If your Webdesign is already dusting and your site regularly brings users to the hair, then you can be relative no matter whether image A or image B arrives better or which button color more often leads to clicks.

In the year 2018 there will be an alarming number of pages on the Web that have basic usability problems, making it more difficult for users to use.

Below you will find a selection of aspects that determine the quality of your online shop. Check your website critically and determine where there might be a need for improvement.


The key question here is: How do you feel about the user experience?

Loading time

Long loading times annoying. And lead to high bounce rates. According to a study by Google , the likelihood of users jumping out increases by 23 percent at just three seconds. After five seconds, the bounce rate already increases by 90 percent, after six seconds by 106 percent.

Reason enough to check your loading times, for example with Google’s PageSpeed Insights .


Almost perfect: Google could still tweak on its mobile version, even if it is “probably unnecessary”. Otherwise, the load times are exemplary – no wonder.

By the way: Long loading times can become a major problem for online shops, especially in the mobile sector. Here, the patience of users is known to be even lower. If a mobile page loads for more than three seconds, 53 percent of visits are cancelled. Here you should, therefore, look very closely (make the best test with TestMySite ) and take countermeasures if necessary.

Mobile optimization

In addition to load optimization, there are many other aspects of mobile optimization, such as user-friendly touch elements, large enough font or the use of responsive design.

Technical problems

If filters are not applied as desired, content is not available, internal links are flawed, or outdated techniques such as frames or flash are used, this is not beneficial to the user experience or SEO.

Clear messages

What is your website about? And what sets them apart from the many others who offer similar products and services? If you do not adequately answer this question for users, you’ll be getting rid of valuable conversions.

Especially on the start page, the visitor should be able to orient himself, what is offered to him on the website and which company is behind it. In my contribution to the user-friendly home page, I explain what you should pay attention to when designing your home page. But also on landing pages, product pages or the About Us page you can and should address the user and show individual team member photo.


At the moment I am looking for a good backpack, in which my laptop fits. I’ve found many shops out there that offer the backpacking model that has made it into my shortlist. But none of these stores stands out so out of the crowd that I would have ordered right away. Somehow they are all not bad, their product pages are meaningful and they offer free shipping. A real, convincing reason, why I should order exactly here and not elsewhere, delivers me but none of these shops.


I keep preaching how important USPs are – and their visualization in the online shop. In the case of the backpack shops, however, the unique selling points are very similar – basically one should rather speak of “equality features” … The question would be, what makes the shops next to the “free shipping” and “purchase on account” really unique?

Content with added value

If the own USPs hardly differ from those of the competition, it would be all the more essential to address the user emotionally – for example, through individual, sympathetic product texts. And to offer them added value, for example through a buying guide with valuable tips on what to look for when choosing a backpack. Or with an article that shows all the advantages of carrying a backpack …

However, none of these shops seems to focus on content marketing – with good, relevant content that could make a real difference in my user experience and prove their expertise!

Web content

Good content can influence the customer journey in your favour. In order for your content to work, it should also be prepared and written in an appealing way.

Pay attention to text fragments such as headings or button labels – are they user appealing? So-called microcontent personally appeals to users on your landing pages, makes them want your products and shows them that they know their needs.

product pages

High-quality images, meaningful product texts, information about shipping and additional costs … you can create a positive user experience on your product pages and thus positively influence the conversion rate.

buying process

If there are problems with the checkout process on your online store, the conversions are quickly over. Not only technical problems can cause trouble here, but above all, lack of transparency or a cumbersome ordering process. What constitutes a user-friendly purchasing process that users do not cancel.

Web forms

One crucial reason why users do not complete an order process is that they have a long, confusing form. If possible, ensure that your potential customers can complete the order form with little effort and in a few steps.

Before A/B Testing: Check usability thoroughly

If you are unsure if your website is sufficiently user-friendly, you can

  • have them tested for UX standards as part of a UX audit
  • or perform user tests to track down sticking points. User tests are also possible with a few users. Note, however, that effort and costs are usually relatively high.

Procedure for A/B tests: You have to pay attention

If your site meets the key requirements in terms of user experience and finetuning, then A/B testing is a very useful method.


If the UX is broadly correct, you can fine tune.

Among other things, you should note the following points so that your A/B test is successful and meaningful:

  • Mastering the Testing Software / Technology: Doing an A/B test? It’s not that easy. In order to get significant and no falsified results, you should run an experienced professional who knows the technique and data evaluation well.
  • Enough time to plan: In addition to the technical aspects, it is also important that the test is thoroughly planned, carried out and evaluated. Defining goals, creating variants of the test object, evaluating results – all this takes time and should not be rushed.
  • Test run time should not be too short: In order to achieve meaningful results, the test period should not be set too short. This is especially important for sites with few visitors and conversions.
  • Try fewer variants than too many: multi-variant tests are not ideal, because they are complex and complex; the evaluation of the results will be difficult.

A/B testing tools

You can use the tools of different providers to carry out A/B tests. Here is a selection of well-known tools for A/B testing:


When it comes to precisely improving details on your online store and finding out what your users value, A/B testing is without question a useful method.

If the conversions on your website leave something to be desired, the bounce rate is high and the User retention rate is low, your first step should be to have the usage experience critically reviewed or checked. This is the only way to find out what the problem is and how you can improve the user experience.

About the author


Vikram is a Digital Media Strategy Consultant who helps small business owners grow their
business. He is passionate about blogging, digital marketing, and emerging technologies.

Add Comment

Click here to post a comment