Two posts ago, I talked about the importance of laying a web analytics foundation for your company by measuring, valuing and analyzing the critical few visitor behaviors on your site that have an impact on your bottom line.  In my last post, I talked about the step after that, which is determine why the data you've collected is the way it is (the why?). Once you've listened to why your customers couldn't complete whatever tasks they were trying to accomplish on your site, you should have a bunch of ideas on how to fix it.

Website Under Construction.jpg

Now it's time to take those ideas and figure out which ones will work the best.  This is done through experimentation and testing.  Lucky for us Web geeks, testing on the Web holds a major advantage over all other marketing channels because of very low cost and fewer limitations.  This has made setting up, running and controlling tests - as well as the ability to collect and analyze the data - easier and cheaper than ever. This in turn makes guessing what you should do with your site  an even worse proposition than guessing what you should do in offline channels (which is also bad).  Not doing experimentation and testing isn't an option.  It's mandatory.

With that said, what can you do?  Testing on the Web basically boils down to two options - A/B Testing and Multivariate Testing.

A/B Testing

These tests are the cheapest and quickest.  You test two uniquely different participants for a predetermined outcome. For example, you make two very different versions of your homepage and see which one is best at getting visitors to click through to the product categories. Or, you see if providing a 2-step checkout process versus a 4-step checkout process works better for your e-commerce conversion rate.

Multivariate Testing (MVT)

With these tests, you create multiple variations of multiple elements of a page and have a testing tool (like Google's Website Optimizer) mix and match them to the visitors. In the end, you are able to see not only which elements contributed the most to a lift for your outcome, but also which combinations of elements work the best.

While A/B Testing is much easier and demands less resources, it does not give you data that is as good as MVT.  Why?  Because after your A/B test is complete, you may know that version B of your homepage performed better, but you don't know why. Was it the headline, the images, the navigation layout, the color of the call-to-action button?  Only with MVT can you find out.  So, start with A/B tests to get your feet wet, but jump into MVT as soon as you're ready.

Get Creative

You can use these tests to experiment with just about anything. Of course, you want to test those areas that will most affect your bottom line. But when you've gotten comfortable with testing, that can get very creative. For instance, let's say you have a $100 product that you are offering on sale for $10 off.  How would offering 10% off instead of $10 off affect sales? You're thinking it won't because it's the same amount? You might be surprised :).

How about testing more cheaply and easily on your site what would cost more money to test elsewhere? Like products you're thinking of putting in your store, calls-to-action for paid search ads, or offers in your email newsletter.

If you're tracking what's important and have actively put in place ways to listen to your customers, now it's time to find out the best ways to deliver what they want. So, use what customers give you to start developing hypotheses about how you could improve their experience. Put your ideas on how to accomplish the goal into action! If you do, you will start to have a distinct competitive advantage.

February 22, 2012

Mike Fleming specializes in Analytics and Paid Search for Pole Position Marketing, a leading search engine optimization and marketing firm helping businesses grow since 1998. You can follow Mike on Twitter at @SEMFlem. Mike enjoys playing, writing and recording music along with playing basketball to get his workout in. He resides in Canton, Ohio with a girl who threw a snowball at him one day…then married him.

Mike and the team at Pole Position are available to help clients expand their online presence and grow their businesses. Contact them via their site or by phone at 866-685-3374.


It takes time to test website content, but the effort is well worth it. It's important to understand target audience behaviors in order to make changes to a site that will result in more conversions. You never know what the best solution is until you test.

Mike, nice article. All interesting stuff. I especially like the "try 10% off instead of $10 off a $100 product." It's all the same, but not to consumers. Good marketers know all about that. A site should play around with pricing numbers in that way, and attempt to keep track of sales differences. A little easier said then done, as there are different metrics that can affect sales, so you can't just quickly assume that one wording change is responsible for an increase or decrease in sales. But it is worth the effort of trying to track these things.

Comments closed after 30 days to combat spam.

Search Engine Guide > Mike Fleming > So You Think Your Changes Will Help Your Customer? Prove It!