Webtrends Optimization

Update: Check out Part 2 on Breaking Down Multivariate Testing and Part 3 on Rules for a Successful Multivariate Test.

Testing is not hard, but there are fundamentals that guarantee a successful optimization campaign.  To help get marketers up to speed with the basics, starting today, I will be writing about one topic per post and put together what I call Billy’s Optimization Guide.

The natural place to start is with a/b split tests, so let’s begin there.

First, a quick useful definition of an a/b split test: the competition of two distinct pages, where a portion of live traffic, usually 50%, is sent to one page and the rest to the other.  The winner is the page that provides the highest conversion rate, or whatever KPI is appropriate.

I want to emphasize that a good a/b split test requires distinct pages.  If that’s too vague, a simple rule that we follow at Widemile is:

You should be able to tell the difference between the 2 pages from 15 feet away.

Anything else isn’t a big enough change to be efficient in a split test and likely should go into a multivariate test.

With that definition in mind, here are three essential types of a/b split tests.  These are three of the tools in the testing toolbox that you should consider when putting together your optimization campaign.

1. Template test: Test the same general content (hero shot, copy, and button color) with a different layout and/or creative treatment.  The goal is to have a new template that better emphasizes the value proposition, improves readability and sets up well for a multivariate test.

   

   Use this when… you want to make sure you have a solid design, before or after testing messaging.  The majority of the time this should be your first test.

2. New concept test: Test a totally new approach.  Don’t let anything hold you back, test what you think will work best and see if it beats the original.  The approach here is to break out of the box and create a page that’s holistically designed around a new marketing concept.  Sometimes this involves introducing new functionality, animation, interactivity and other dramatic steps.  However it can also be on the smaller scale, such as introducing new messaging that requires a complete redesign.

   

   Use this when… your current page has already been tested many times and beating it has become difficult or you believe the way to really grab your visitors is through a big change.  This should only be done when the benefits of multivariate testing (knowing individual factor influences) are outweighed by the possible gains.

3. Funnel test: Send users to different multi-page experiences, e.g. no registration vs. requiring registration (below) and a one page form vs. a 3 page form. A funnel test can also be done with a multivariate but is simpler as an a/b split test.

   

   Use this when… you want to test content that extends past one page.  This should be done earlier in the testing process so that you don’t end up optimizing a page and then find out it’s a suboptimal experience.  It can be more technically demanding to do this sort of test though.

Every optimization campaign is different and so knowing what kinds of tests are available is one of the most important places to start.  For my next post, I will talk about the different ways to use a multivariate test.  Please post in the comments if you have any questions or contact me via Twitter @billysblog.

I try to keep this blog strictly about testing, but I have to make an exception. One of the biggest reasons I am working at Widemile is because I believe it has a great future ahead of it and our announcement today is part of that future.

There are two exciting parts to the announcement: the first is that we are revealing our new optimization platform, which has been designed with agencies in mind, and second, we have 13 partners lined up to begin using our new optimization platform. That list is comprised of these amazing companies: Ascentium, Avenue A | Razorfish, Brand Digital, Closed Loop Marketing, DDB in Seattle, Palazzo Intercreative, POP, Portent Interactive, Red Bricks Media, SolutionSet, Stratigent, TMP Directional Marketing, and ZeroDash1.

In celebration of this announcement, I have moved the blog to this domain, testingblog.widemile.com and redesigned the page to look more like our new website at www.widemile.com.

This is the start of a new era for Widemile and as my company matures, I plan to continue to blog and become an even more valuable resource to the online marketing and campaign optimization community.

Press release @ Widemile.com. 

Official press release in PDF format.

I love web designers and creative people. They create beautiful things out of words and ideas we give them, yet still have to deal with people judging every little thing they put out into the world. Working on company websites is especially an ordeal. Everyone has an opinion, from CEO to marketer, but really who knows what will work?

Do you go by experience, seniority or even… best debating skills? Usually seniority wins out and one person makes all the defining decisions.

So why test? At its core, testing is about finding out what works best, without any of the politics. If I put up two versions of a page and one gets more sales than another, its hard to argue with that. My job ends up proving people right or wrong, judged by what real visitors are telling us.

Test to:

• Know the impact of redesigns (quantitative)
• Figure out what your customers are and are not looking for (qualitative)
• Find the truth
  

As a marketer, our goals are to speak to our audience and get them interested in our business. With testing, marketers have a way to directly listen to what visitors want, to find out their language. So start testing to start listening.

Make sure you come back for more, I will be writing more on reading messaging through testing. This is just the beginning!

Being a newcomer to the small but growing online testing industry has put me in a unique position. Not only do I get to learn about completely fresh and exciting marketing technology, but also I have an opportunity to build upon this nascent industry. While I have only just started, already I am way ahead of most online marketers in terms of understanding online content optimization.

Recently, A/B, split and multivariate testing have become hot topics in online marketing, yet they are still far from commonplace. With that comes a lack of resources or even knowledge about testing, although the demand to know more is quickly rising. As I have found out, and hope to share with this blog, there is a lot more to testing than just trying new things out. Much like the many businesses that have been burned by bad PPC campaigns, there are businesses that have had bad experiences testing. However, I know for a fact that good testing methodology and technology drives results. I hope that I can help other marketers out there learn more about optimization and develop a culture of testing and optimization in their companies.

Don’t just change it, test it.