Webtrends Optimization

It’s been a fun journey here at Widemile and although Widemile, the name, is disappearing, our mission to bring optimization to the forefront of online marketing has not ended.  I will be moving this blog over to the Webtrends domain very soon, for now, please see my post on the Webtrends blog for my thoughts on the acquisition.

Thanks for being a loyal reader of my blog, I will be posting a lot more once I switch over to Webtrends, so don’t forget to check it out.

-Billy

Webtrends is hosting a completely transparent marketing campaign, called The Open Campaign.  In the campaign, Webtrends and a few partners, including Widemile, are conducting their various specialties and sharing the results with the public.  The idea is to let you all learn about all the great marketing technologies out there and how it ties together with Webtrends, as well as seeing the performance of the campaign itself.

Check out the Site Optimization section where I will be discussing the test we designed for The Open Campaign and the results.  See for yourself exactly what its like to do optimization with Widemile Optimize and how my brain works throughout the process.

Update: Check out Part 2 on Breaking Down Multivariate Testing and Part 3 on Rules for a Successful Multivariate Test.

Testing is not hard, but there are fundamentals that guarantee a successful optimization campaign.  To help get marketers up to speed with the basics, starting today, I will be writing about one topic per post and put together what I call Billy’s Optimization Guide.

The natural place to start is with a/b split tests, so let’s begin there.

First, a quick useful definition of an a/b split test: the competition of two distinct pages, where a portion of live traffic, usually 50%, is sent to one page and the rest to the other.  The winner is the page that provides the highest conversion rate, or whatever KPI is appropriate.

I want to emphasize that a good a/b split test requires distinct pages.  If that’s too vague, a simple rule that we follow at Widemile is:

You should be able to tell the difference between the 2 pages from 15 feet away.

Anything else isn’t a big enough change to be efficient in a split test and likely should go into a multivariate test.

With that definition in mind, here are three essential types of a/b split tests.  These are three of the tools in the testing toolbox that you should consider when putting together your optimization campaign.

1. Template test: Test the same general content (hero shot, copy, and button color) with a different layout and/or creative treatment.  The goal is to have a new template that better emphasizes the value proposition, improves readability and sets up well for a multivariate test.

   

   Use this when… you want to make sure you have a solid design, before or after testing messaging.  The majority of the time this should be your first test.

2. New concept test: Test a totally new approach.  Don’t let anything hold you back, test what you think will work best and see if it beats the original.  The approach here is to break out of the box and create a page that’s holistically designed around a new marketing concept.  Sometimes this involves introducing new functionality, animation, interactivity and other dramatic steps.  However it can also be on the smaller scale, such as introducing new messaging that requires a complete redesign.

   

   Use this when… your current page has already been tested many times and beating it has become difficult or you believe the way to really grab your visitors is through a big change.  This should only be done when the benefits of multivariate testing (knowing individual factor influences) are outweighed by the possible gains.

3. Funnel test: Send users to different multi-page experiences, e.g. no registration vs. requiring registration (below) and a one page form vs. a 3 page form. A funnel test can also be done with a multivariate but is simpler as an a/b split test.

   

   Use this when… you want to test content that extends past one page.  This should be done earlier in the testing process so that you don’t end up optimizing a page and then find out it’s a suboptimal experience.  It can be more technically demanding to do this sort of test though.

Every optimization campaign is different and so knowing what kinds of tests are available is one of the most important places to start.  For my next post, I will talk about the different ways to use a multivariate test.  Please post in the comments if you have any questions or contact me via Twitter @billysblog.

I will be at ad:tech New York on November 3-4 manning the Widemile Booth #922.  If you have any questions on optimization, landing pages or anything else, stop by and we can chat.

Also, I will be giving some mini-clinics on optimization best practices, how creative influences optimization and perhaps a case study or two based around those topics.  The last half of those clinics, I’d like to take a look at some live pages and review them.

But to be more effective, I’d like those live pages to be from volunteers.  If you have a page you’d like me to look at, send me a URL and a good time for you and I’ll try to make sure you get included in a clinic around that time.

Find me at ad:tech booth #922

If you catch me when I’m free, I also can take a look at your page and give you some of my thoughts on how to improve and test it.  Send me your URL’s beforehand for best results though.

Hope to see some of you there!

I was on vacation and have been swamped in catching up with client work, but I promise I’ll finish my article on fractional factorial test design soon! It’s a very long article and I want to make sure I get it right.

Hope some of you got to see the Widemile booth at SMX Advanced in Seattle (I didn’t since I was on vacation unfortunately.) I heard good things about the panel Widemile’s CEO, Robert Bergquist, was on and that Jonathan Mendez was kind enough to recommend my blog as a resource, so thanks goes out to him.

Thanks for sticking with me!

I’ve written an extended definition for interactions in preparation for a long post about full and fractional factorial. Understanding interactions is a critical part to understanding full and fractional factorial also. I look forward to clearing up some misconceptions about fractional factorial test design. Hopefully I’ll have the post done next week.

CC photo credit: Zeno_

Someone at the office has put together a great glossary, so I modified it slightly and have posted the glossary as its own page (it has a tab dedicated to it above now.) It is in a usable but not optimal form right now, so I’ll be updating it every now and then. I have also decided to add expanded definitions, in the form of separate pages dedicated to a single word. The first word to be done was “experiment.” Please check it out and let me know what you think.

In the past, I have stepped away from using technical language and jargon but, with this glossary, I will begin using the language I use at the office. My hope is to acclimate others and help them understand the terminology used by myself and others at Widemile and around the industry.

If any of you use Twitter, I’ve opened an account, Billysblog. I’ll be posting interesting links related to optimization, marketing and Widemile, as well as any thoughts I have about online marketing. I’ve included a feed on the left nav bar of this blog too.

I try to keep this blog strictly about testing, but I have to make an exception. One of the biggest reasons I am working at Widemile is because I believe it has a great future ahead of it and our announcement today is part of that future.

There are two exciting parts to the announcement: the first is that we are revealing our new optimization platform, which has been designed with agencies in mind, and second, we have 13 partners lined up to begin using our new optimization platform. That list is comprised of these amazing companies: Ascentium, Avenue A | Razorfish, Brand Digital, Closed Loop Marketing, DDB in Seattle, Palazzo Intercreative, POP, Portent Interactive, Red Bricks Media, SolutionSet, Stratigent, TMP Directional Marketing, and ZeroDash1.

In celebration of this announcement, I have moved the blog to this domain, testingblog.widemile.com and redesigned the page to look more like our new website at www.widemile.com.

This is the start of a new era for Widemile and as my company matures, I plan to continue to blog and become an even more valuable resource to the online marketing and campaign optimization community.

Press release @ Widemile.com. 

Official press release in PDF format.

Welcome! I just got word that I was featured in MediaPost’s Around the Net in Search Marketing. If you aren’t sure where to start, check out my organized lesson guide. Also, I have attempted to organize everything in appropriate tags and categories found on the right navigation bar.

I just started this blog a few weeks ago, so there’s going to be a lot of growth and new posts, so if you like what you see, subscribe via RSS or get e-mail updates.

In addition, if you have any questions, opinions, or suggestions on the site or my posts, feel free to leave a comment.