Note: This is the second post of a 3 part series, each focusing on one type of test result that is tough to deal with. Read the first article on highly mixed data.
As an optimization analyst, this is probably the hardest result to bring to a client. Oddly enough, it actually is favorable to part 1′s highly mixed data and part 3. I am talking about optimization that determines that the original page is better than the tested variations.
How does this happen?
Sometimes a page just gets it right. How would you change Google? I looked for a few variations and came across one by Andy Rutledge and another by Valacar. They both are beautiful designs and a lot of thought were put into them, but at the same time, would they really make Google more profitable? It’s definitely a tough sell and there is a big challenge in improving this type of page.
The goal is for users to search. Yes, they want users to click on ads eventually, but there’s not a whole lot they can do for ad clicks on the homepage. The best they can do is get users to search as fast as possible. So would a redesign make it more usable and readable? Maybe. To a level that it would increase their revenues? That’s tough to say.
The more simple the goals of the page, the less information and messaging the users needs, the more likely that the page will be difficult to optimize.
What can you do to prevent this?
Be careful when choosing a page to test. Find a page where the user will take some time to look at what is going on. This is another reason why most landing pages are great places to optimize, because users naturally need to be introduced to the product and sold on why to convert.
The logical thing to do would be to simply refrain from testing pages that seem to be performing well, but this is rarely a good rule. Unless it is performing well because of a lot of testing, then you don’t really know if a page is performing well or not (see my post on conversion rates.) Testing always brings surprises and personal judgment is no replacement for a test; a good looking page can perform poorly and a page with subpar creative can perform great.
What can you do if this happens?
Because of the above reasons, you may actually plan for this scenario to occur. Many people believe redesigning an old page will provide improvement, but what if it is old and performing well? In that case, you may plan to try to improve but not expect to beat the old version.
In any case, if your original page wins, then you have confirmation of your page’s success. It is unlikely that all possible improvements were tested in one test run though, so it may take a few more runs to really confirm its solidarity, but the page has won against the initial best ideas and that is an achievement.
This lesson tells you that you can move on and that is progress in itself.
Moving forward, I would try drastically different approaches, either in layout or design and testing around offers. Otherwise, I would apply the successful original page to tests for other areas of your site.
I have to be honest when I say that this rarely ever happens. Almost every page has room for improvement at every step of the conversion funnel.
Whew, I will try to get the third and toughest optimization result next week.