Web of Data

March 7th, 2009 | Jascha Kaykas-Wolff

More platforms and more applications come online everyday. We know this. They all create data. We know this. Our businesses collect data, it comes at us from everywhere and from every channel we operate in. We know this too.

NYTimes News Data

In truth we are attacked by data. The sheer volume that is available. Some of us run from data. Some of us stay afloat in data. Some of us channel our data into value. Regardless, we are surrounded by data.

The Library of Congress with 130 million items on about 530 miles of bookshelves, including 29 million book, 2.7 million recordings, 12 million photographs, 4.8 million maps, and 58 million manuscripts, can be stored on 10 terabytes.

IDC says that 161 exabytes of digital content was created, captured, or replicated in 2006. That’s 161 billion gigabytes, for those of you counting (I lost count at 6,979,563). By 2010, IDC suggests that number will increase sixfold to 988 exabytes.

To put that in perspective, in 2006 alone we created 16 million times more data than exists in the whole Library of Congress. In 2010 we’ll create 96 million times more data than the LoC.

What is all of this data? It’s memories like photos, documents, and videos. It’s communications like emails, phone calls, IMs, Facebook messages, and tweets. It’s business operations customer campaign management, general ledger, CRM and so much more. Is it fluid? Yes. Always? No. Some of it is purely for storage to comply with omnipresent edicts like Sarbanes-Oxley, PCI compliance, etc.

It is not humanly possible to parse all of this data; and to be clear some of it shouldn’t be. Our only way out of this web of data is to embrace it. We need software to help us make sense of it. We have to embrace its complexity, gain insight from it, and take action with it.

Fifteen years ago when WebTrends entered the web analytics space, the data we captured and processed was about onsite activity. Over a decade ago this was revolutionary and it has been largely manageable since. However, today’s enterprise businesses, of which mine is one, have our own web of data. It’s comprised of CRM, BI, BPM, social media monitoring, and more. The business ecosystem necessitates an ability to plumb together this data and expose new patterns, insights, and more valuable actions from the interconnected data.

Interconnected data is part of our Open Exchange focus and there are more details to come at Engage. See you there.