As an optimization analyst, this is probably the hardest result to bring to a client. Oddly enough, it actually is favorable to part 1’s highly mixed data and part 3. I am talking about optimization that determines that the original page is better than the tested variations.

How does this happen?
Sometimes a page just gets it right. How would you change Google? I looked for a few variations and came across one by Andy Rutledge and another by Valacar. They both are beautiful designs and a lot of thought were put into them, but at the same time, would they really make Google more profitable? It’s definitely a tough sell and there is a big challenge in improving this type of page.

The goal is for users to search. Yes, they want users to click on ads eventually, but there’s not a whole lot they can do for ad clicks on the homepage. The best they can do is get users to search as fast as possible. So would a redesign make it more usable and readable? Maybe. To a level that it would increase their revenues? That’s tough to say.

The more simple the goals of the page, the less information and messaging the users needs, the more likely that the page will be difficult to optimize.

What can you do to prevent this?
Be careful when choosing a page to test. Find a page where the user will take some time to look at what is going on. This is another reason why most landing pages are great places to optimize, because users naturally need to be introduced to the product and sold on why to convert.

The logical thing to do would be to simply refrain from testing pages that seem to be performing well, but this is rarely a good rule. Unless it is performing well because of a lot of testing, then you don’t really know if a page is performing well or not (see my post on conversion rates.) Testing always brings surprises and personal judgment is no replacement for a test; a good looking page can perform poorly and a page with subpar creative can perform great.

What can you do if this happens?

Because of the above reasons, you may actually plan for this scenario to occur. Many people believe redesigning an old page will provide improvement, but what if it is old and performing well? In that case, you may plan to try to improve but not expect to beat the old version.

In any case, if your original page wins, then you have confirmation of your page’s success. It is unlikely that all possible improvements were tested in one test run though, so it may take a few more runs to really confirm its solidarity, but the page has won against the initial best ideas and that is an achievement.

This lesson tells you that you can move on and that is progress in itself.

Moving forward, I would try drastically different approaches, either in layout or design and testing around offers. Otherwise, I would apply the successful original page to tests for other areas of your site.

I have to be honest when I say that this rarely ever happens. Almost every page has room for improvement at every step of the conversion funnel.

Whew, I will try to get the third and toughest optimization result next week.

CC photo credit: philosophygeek

Alright, so you’re ready to test. You’ve got tools and the skills to design a test. But when optimization begins, where should you start? While we all would love ROI to be the only driving factor in optimization, your resources and reach usually dictate what you end up testing, as well as ROI.

Here’s what I think about when looking for candidate test pages:

1. Importance: Is this the best page to accomplish your goals? Take a look at your overall marketing campaign and see how important this is to the whole process. Look at drop-off points and find pages that are important but weak links.
   
2. Technical: Will this page be easy to optimize? How much technical involvement will it require? If you’re testing dynamic elements you may need some additional help. Maybe you can test a page outside of the development schedule or on a separate server. Look for pages that have less restrictions and can be modified quickly. Also examine the page for what can’t be tested and what you may want to test on a page. Some tests are harder to create than others, both technically and creatively.
3. Goals: What are you optimizing for? Pages with one goal are easier to optimize since you can drive everything on the page towards achieving that one goal. If you think a page is under performing, then it may be an easier page to optimize also. Lastly, think about how easy that goal will be to measure. If there are multiple conversion possibilities or the conversions are offline, it will be difficult to test.

In the end you are asking a multi-part question: Will a lift here be more valuable than a smaller/same/larger lift elsewhere that will take X amount of work and time?

Just remember, you can always test a page later on, even if it may not be the best candidate now. If you have the resources you can test pages simultaneously too. Just make sure they don’t impact each other in any way, so as to not skew your results.

As your optimize more and more, it will be harder to choose, but that’s a good sign. It means you’ve got a lot of great pages and that is what you want testing to do for you.

Photo Source (under CC)

Unfortunately, many projects are started without tangible metrics to strive for other than finishing it on time and within the budget. One part of being a marketer, or any businessman or woman, is showing that what you do is valuable and has a solid number attached to it.

Because of that, every marketer should make a resolution to measure and test their campaigns rather than just finishing them.

Achievements like site redesigns and campaign launches should be celebrated, but they also should be effectively tracked and analyzed from beginning to end.

Jason Burby wrote a fantastic article at ClickZ about ringing in the new year by defining and executing based on goals for 2008. In it he describes how sometimes projects succeed in some measures but fail at others, so overall is it a success?

He even mentions multivariate testing as part of the solution: “One part of this new way of thinking is to ensure you have a simple, easy to use testing platform to try different things and to measure the impact of tests based on goals. If you haven’t invested in an A/B or multivariate tool, 2008 is the time.”

What I’ve found in doing optimization and working with marketers in web analytics is that defining success in every campaign is key to being a quality marketer. And the only way to know if you met those goals is to measure what you do and how much it drives those goals.

For web page testing, the goal typically is to raise conversion rates and so its easy to keep that in mind. However if I never changed conversion rate as my ultimate goal, I might miss out on other valuable goals. A great example is if there are two pages with different subscription levels for a product, then possibly the lower converting page has a higher lifetime ROI.

The great side benefit of having a goal and measuring what gets you there, is that you can learn from success and failure. Even when I test a page and get an improvement, I exam what factors were the worst, along with what did best. It tells me what scares away and attracts customers so that I can look out for those things in other parts of the page and overall funnel.

Never assume that changing or making a new version of anything is always better. Figure out what you want to improve, make the changes and see if you find that improvement. As long as you have a measurement for success and take the time to track your progress, you can learn from your campaign and in the end, get closer to reaching and beating your goals.