Big Data Hype, Supernovas, McKinsey’s Latest Survey, and AIIM’s Digital Landfill Dive
A few years ago, Terence and I were trying to get our arms around the world of big data and how to effectively communicate its size and challenges as we were talking to analysts, media, prospects, partners, and, of course, the venture community. Hopefully, most of our readers would acknowledge that we have a fine grasp of language (we like to engage in a Scrabble battle of wits upon occasion) but I must admit that we have been eclipsed by many in the race to illustrate just how “big” big really is. My favorite is this:
“We are engulfed by a supernova of data.”
Now, Webster’s defines supernova in two ways:
“1: the explosion of a star in which the star may reach a maximum intrinsic luminosity one billion times that of the sun or 2: one that explodes into prominence or popularity.”
To be honest, I can’t quite decide whether the author was going for definition one or two because both could apply. Definition one applies to size and you could make a case that definition two applies to hype. Certainly, one of the companies that occupies our space (Cloudera) has come under fire recently for—how do I put this?—winning the 2012 hype awards. As a marketer, this is not an award you want to be nominated for, let alone win, but it certainly illustrates Gartner’s hype cycle and how it can be applied to big data. As I stated in a post last year:
“In my mind this perfectly positions the [big data] industry as moving through the Trough of Disillusionment and moving up towards the Slope of Enlightenment… where more companies are seeing the benefits of big data analytics and starting to engage in projects around it.”
Cloudera is one of those early stage vendors that can’t quite deliver on the promise of big data, particularly as it applies to mass adoption where tools and applications—not technology alone—are needed. It is guilty of marketing to the promise of big data without having the tools and solutions to back it up. Or as I like to put it: lots of talk but little substance.
That being said, let’s not confuse hyperbole with company adoption rates and big data challenges. Last year, I wrote a series of posts on McKinsey’s report, “Big data: The next frontier for innovation, competition, and productivity” (just click on the McKinsey cloud tag if you’re interested in reading them). In the report, McKinsey looked at ways big data could be leveraged within specific industries (and across industries with location data) and the value that could be derived in the form of billions and billions (and billions) of dollars. McKinsey’s latest survey on “Minding your digital business” looks at how executives expect new digital technologies to change the way they do business while assessing whether they are capable of dealing with the challenges that result. While expectations are high, it’s safe to say that adoption varies:
“With big data, 49 percent say their companies are focusing their efforts on customer insights, segmentation, and targeting to improve overall performance; an even higher share say their companies should focus their efforts on using data and analytics to generate these insights… Yet just one-fifth say their organizations have fully deployed analytics to generate insights in one business unit or function; only 13 percent use data to generate insights across the company.”
Put simply: while most executives see the value of big data and analytics, few have embarked on big data projects due to a lack of funding and talent (one of the survey’s key takeaways). Certainly, this is no surprise to us because, until recently, big data initiatives could cost millions of dollars as pointed out by Thor Olavsrud in CIO:
“Big Data is a powerful lure, promising to turn the massive and ever-increasing volumes of data inside an organization into a pool of intelligence that promises deep, actionable insight into every aspect of a business. However, that lure can lead you into an expensive trap if you don’t plan carefully… Jeff Muscarella, IT spend management consultant with NPI Financial… warns that Big Data projects can easily ring up seven-figure price tags after you finish paying for the hardware, software and services, and sometimes the glowing business cases presented by vendors lose their luster when you look closely.”
Yes, perhaps Cloudera’s “glowing” big data business case (with a price tag to match), amongst others, is somewhat responsible for the gap between perceived value and real, measurable results. Certainly, AIIM’s (The Global Community of Information Professionals) report on “Big Data: Extracting value from your digital landfills,” confirms that while users see big data’s potential benefits they are also facing significant challenges:
- 70% of those surveyed believed that there is a big data killer application that would be “very useful” or “spectacular” for their business (like—hmmm, I’m not going to tell you my great idea just like the majority of those surveyed!).
- 61% of those surveyed believed that it would be “very useful” to link structured and unstructured data sets (like stock ticker data with twitter streams).
- 56% of those surveyed believed that it would be “very valuable” or “hugely valuable” to be able to perform sophisticated analytics on unstructured content (like looking at the effect of company mentions in Twitter on the company’s stock price as we have shown in this video that showcases our KPI tool).
More startling, 70% of those surveyed believe that it’s “harder” or “much harder” to search for information on their own internal systems than it is to search for information on the web. I am sure that at one time or another we’ve all imagined what it would be like to google our company assets (in lots of siloed databases that contain structured and unstructured content) in the same way that we google the Internet (fast and easy) but this is still not the case. However, we certainly believe that google-like search is a must have for an analytics application as it is one of many standard features in our AnalyticsPBI and FinancePBI applications (yes, a marketing plug but give me some points for the absolute lack of hype). AIIM sums it up this way:
“The range of available analytic techniques across unstructured and semi-structured data is considerable, and this, combined with the need to present a unified data connection across diverse datasets, presents a challenge for most organizations. Expertise is scarce, and generic analytic product platforms are in their infancy, with most organizations resorting to a mix of open source products, in-house development, and best-of-breed applications. For many, there is also a security issue of both a competitive and a compliance nature… However, most of our respondents consider that there are one or more killer apps for big data in their organization that would provide a dramatic competitive benefit, or would provide much better prediction and prevention for business continuity.”
Killer apps aside, there are many benefits to big data initiatives in terms of process improvement, monetary gains, and cost savings. But, as my esteemed colleague (that would be Terence) posted recently:
“Look, big data will not “change the world” until anyone with a college level understanding of statistics can derive useful information from large data sets. After all, it’s not about the data itself. It’s about analytics and the insights those analytics enable you to make. While big data projects propelled by those rocket scientists programmers are nice and all (and of course have solved some very interesting problems), that doesn’t change the fact that the current set of big data tools are inaccessible and unusable by the majority of public and private organizations.”
It’s time to move away from the hype-mobile and start working on providing businesses with answers based on insights derived from data. These folks need tools that they can use today with price tags that don’t break the bank. If you happen to be one of them, take a look at what we have to offer: enterprise-wide big data applications that make big data analytics a reality for organizations and businesses of any size. Not a whole lot of hype going on at PatternBuilders but certainly a lot of substance!