Throughout my career in web analytics I have had the opportunity to work on web projects for some of the largest and most well-known websites in the world, and I’ve been able to do this from many different perspectives, including client-side, vendor-side, and advertising agency-side.  While I expected many of the websites to be slow in adopting some of the more advanced web analytics topics such as behavioral targeting, multivariate testing, and tagging, I didn’t expect for most of them to have the challenges with ongoing performance reporting and representation of data that they did.

These companies all struggled with most of the aspects related to putting together a dynamic and effective monthly report: extracting accurate data from their measurement tools, data organization and visualization, report formatting (including delivery vehicle), and analysis/insights/recommendations.  Most of them were decent with analysis; however, because everything else took so much longer than it should they didn’t have the proper amount of time to spend on analysis, so even that suffered.  Fortunately, the solution to this is relatively simple in explanation and not much more complex in execution: data quality, automation, and a consistent, accessible report format – what we call an Integrated Digital Marketing Scorecard.

At this point you’re probably making a decision as to whether or not you should read the rest of this article – so I’d like to give you a few reasons to continue reading.  While this topic has been written about and discussed quite a bit, I feel that it still has yet to sink in because I’m not seeing it take hold in the work that I do.  And while I don’t profess to be a true EXPERT at anything, I am an intelligent, logical, and creative thinker and like to think that I just get it (and I been practicing Web Analytics for 6 years).  And as previously noted, I’ve had the opportunity to do this type of work for lots of different websites from a multitude of perspectives – the most helpful being my time at a very large and well-known digital advertising agency.  Many of the lessons I learned I did while managing Measurement Operations at that agency.

To formally define the problem, analysts are spending a much larger % of their available time extracting and formatting report data and not enough time on the analysis, insights, and recommendations – where the true utility of the data comes into play as it is converted to decision-making information.  In my rough estimation analysts are spending about 75-80% of their time on the tactical reporting work and only about 25% of their time on the real analysis work.  In a perfect world this would be reversed – spending just 25% of their time on tactical work and 75% on analysis.

As mentioned, the solution that we have come up with to address these problems is what we call an Integrated Digital Marketing Scorecard (or IDMS).  “Integrated” because it is a single report that includes data from multiple data sources, “Digital Marketing” because it contains data from all online marketing and website measurement systems, and “Scorecard” because it is a visual display of key performance indicators on a single screen, providing user-driven comparison and progress metrics and used to make management decisions.

Let’s look at each of the problems individually, as well as how they are addressed with the IDMS…

Extracting accurate data from their measurement tools

At one point not too long ago in my Web Analytics career I was managing the monthly reporting for nearly 15 separate websites, all of which wanted their report as close to the beginning of the month as possible.  Each of the sites had their own set of complexities and issues: tagging problems, custom report issues, communication breakdowns with clients, etc.  There always seemed to be some issue with some data point each month that meant I could not fully automate my data extracts because I had to scrub the data in some way.  This is where Data Quality comes in.  At the core of any good, automated reporting solution is high quality data.  It can sometimes take a good amount of money and/or resources and certainly process changes to achieve the quality of data needed to support this style of reporting, but from experience I can say that it will pay for itself in time saving and decision making support.

In this role I was so incredibly busy during the first few weeks of the month that I chose to only partially automate my data extractions and pull the remainder of the data manually.  I justified this by telling myself that it would actually take longer to automate the query for a data point than to pull it manually.  I also failed to consistently document how I would manually derive a certain data point and would need to reconstruct it and compare it to the previous month’s number to make sure I did it correctly.  This was incredibly time consuming.  This speaks to the second component of the solution, automation.  The bottom line is that if the calculation of a data point can be done fully automated, always do it.  It will save time and reduce the occurrences of errors compared to pulling the data manually.  When we build an IDMS for a client our goal is for there to be as few steps as possible each month to update the data and populate all charts and graphs.  In many cases we’re able to get this down to 4 steps: select a new month, click button to run queries, validate new data, click button to update tables/charts/graphs.

Data organization and visualization

Analysts struggle to find the best way to organize, visualize, and present the information in their reports.  It is very rare that I’m impressed when an analyst shows me a report they authored that they feel is one of their best pieces of work (by the way, don’t bring an example report to an interview unless it looks like something you would see on TV – you know who you are).  It amazes me the lack of attention to detail that is pervasive across the industry.  Presentation of the report can, in some cases, determine its success or failure.  If you have bad data to report, at least make sure it is presented professionally.

There are a few guidelines we follow when organizing and visualizing the data in an IDMS:

• Organize the data in a tabular or navigational format so that the user can page through different sets of data while remaining on the same screen.
• Group similar data points together into the different tabs and have them follow some type of logical progression (i.e. Acquire-Convert-Retain)
• Show a minimal amount of data on a screen, but allow the viewer to interact with the report to show historical data and compare it against more recent data.
• Only show one or two data points on a graph that are related – don’t overcomplicate it with 3 dimensions and multiple metrics, etc.

Report formatting (including delivery vehicle)

It can take a very long time to manually update all of the charts and graphs and screenshots necessary for a given report, we get around this by extending the automation to include updating the charts and graphs in the IDMS, not just run the queries.  Most people don’t fully understand the formatting capabilities of Excel but there are a lot of fancy things that can be done with formatting and interactivity – especially in Excel 2007.  Have you ever heard of the camera function?  If not, Google it and I’m sure you’ll find a great use for it.

Speaking of Excel 2007, that is our preferred vehicle for delivering monthly reports.  It supports the tabular navigation previously discussed, all historical data can be included within the file in hidden sheets, the Scorecard can be made to look like a polished web application, and most importantly, it’s in Excel – something that everyone has access to. Alternatively, tools like Excelcius can be used to build this same type of functionality but publish it as an interactive Flash object embedded within a PDF file.  I also love the idea of delivering these via the web, but that would require some web development because as of yet there is still not an available tool that I know of that can WYSIWYG a Scorecard.

There are quite a few other report formatting topics that I could discuss, but to keep this relatively readable I’ll add just one more that truly sets the IDMS apart from other solutions – specific real estate is set aside in the Scorecard for the analysts’ insights, observations, and recommendations.  This section updates when the viewer pages through the different tabs in the Scorecard so that it corresponds with the data they are viewing.  This is where data is turned into information that can drive decisions – without this analysis a report is not incredibly useful for obvious reasons.

The best way to approach writing the commentary is to first understand the business questions that need to be answered every month.  The questions shouldn’t simply be worded versions of KPIs like “How many visitors came to the site this month compared to next month?” – they should be business questions like “Which media tactics drove the highest engagement with product content?”  These are the questions that will be answered in the analysis section every month, supported by the KPIs and data in the Scorecard.

If you made it through this article, THANKS FOR READING.  I sincerely hope that it was worth the time you spent reading it.  If you’d like to chat further about this topic or see some screenshots of our Integrated Digital Marketing Scorecard, please feel free to reach out to me via email at Jason.Widup@WebTrends.com.  Happy Analyzing!

*originally posted on IQ Workforce

I love the question, “How Do You Measure Success”. It is the very first question I ask my clients when I start working with them. How do they measure success, how does their company, department or the various teams within their organization measure success? Those answers provide the base that I use to build their metrics upon. It is common to hear, “more traffic”, “increased revenue” and “customer engagement” as some of the answers. Each gives us a place to start and we can go from there.

As a consultant it’s just as important for me to ask myself the same question as I start each new project, which I share at my initial meeting with the client. For me the measure of success as a consultant is when my actions, recommendations, support and expertise leads to a measure of success for my client. I want my answer to help set the bar for our working relationship because if I am not helping them to increase their ability to be successful then I am not living up to my own expectations.

I believe that when you are measuring for success you are setting yourself up to win. This is one of the reasons I had decided the question “How Do You Measure Success” would be a good title for a post about KPI’s, because KPI’s are just that – measurements used to quantify success. Success of course is the most positive measurement of results and that’s what executives, management and team leaders are looking for – results.

This is a great question to ask the various business users during the Business Requirements Gathering phase. The key point in analytics isn’t gathering the data. The key point of analytics is how we use the data we collected to make decisions. Nearly every analyst I speak with at one time during a conversation about analytics will make a point about wanting their business users to actually use the data to drive their decisions. Analysts don’t see the production of reports as a measurement of our success.

I’m going to be asking more people to answer that same question – my coworkers,  WebTrends customers, business contacts and other people within the industry. This will be a regular topic/on-going series and I look forward to sharing their responses with you.  Of course you can join in right now and  share your answer with us  “How Do You Measure Success?”

“Okay, so what do all of these numbers mean?”

The topic that always follows and is extremely popular in the realm of web analytics are key performance indicators, or KPIs. While they certainly have their place in not just web analytics but all metric based initiatives, they are very often used as a solution to the problem rather than a tool.I am often asked what “good KPIs” are and my response is always the same. Good KPIs are simply the correct KPIs that are actionable and easy to understand.

To prevent unactionable and confusing inefficiencies caused by KPIs, I would urge everyone to begin (or take a step back) by asking yourselves a simple question:

What is the main purpose for measuring our digital activity?

While every measurement program has differing objectives, here are some key areas to defining your measurement program’s strategy:

1. Objectives
   1. What are your organization’s objectives?
   2. Who are the stakeholders for this measurement program and what are their objectives?
   3. What are the objectives of your website on a high-level and within each functional area?
2. Decision making
   1. Who will be responsible for decision making within this measurement program?
   2. Existing measurement programs
   3. What reporting strategies are already in place?
   4. What existing report data is actionable?
   5. What else can you learn from existing reporting?
3. Benchmarking
   1. How will you define benchmarking? Against competitors? Against existing data? After setting baselines?
4. Measurement frameworks
   1. How will reports, actions and insight be delivered to each area of your business or stakeholder?
5. Analysis
   1. What level of in-depth analysis will exist within your measurement program?
   2. What is the framework for analysis?
   3. How does the decision making process differ for this level of insight?

Only once you understand your measurement program at this level can you begin to identify KPIs and their action plans. While this material above may be second nature for WA experts and practitioners in the field, I urge everyone to step back for a moment and identify the gaps in your measurement program. 

I’m very interested to hear from you on this subject (contact me directly if you’d like to discuss, but want your comments kept private)  What are your answers to these questions: 

1. Have you spent the time upfront identifying a strategic measurement program?
2. Where are the gaps in your measurement program?
3. Is your organization successful in measuring online activity? How do you know?
4. What are other important aspects of a measurement strategy?