Gartner, Hype Cycles, and Big Data
Greetings all! While we’ve been super busy at PatternBuilders working on a destination application that we are all very excited about, doing some development work, talking with potential partners and prospects, AND not to mention the fact (but I will) that Terence and I are getting close to the finish line on our book, I came across this interesting article on Gartner’s hype cycle and the recent addition of big data to it.
Now, I am pretty sure that you all are familiar with Gartner’s magic quadrant methodology that essentially evaluates all the particular technology players in a specific industry across four quadrants: challengers, leaders, niche players, and visionaries. For those companies looking at vendors for a specific solution, the magic quadrant can help them understand how they stack up against each other. For the vendors, it’s an opportunity to take an objective look at the industry as a whole and understand what they do well and where they could be better.
Gartner’s hype cycle has been around since 1995. It is their take on where emerging industry/trends fall in the cycle of:
“… over enthusiasm, disillusionment and eventual realism that accompanies each new technology and innovation. The Hype Cycle Special Report is updated annually to track technologies along this cycle and provide guidance on when and where organizations should adopt them for maximum impact and value.”
The hype cycle “graphs” (well, sort of but more on that later) what phase of hype an emerging technology falls into before (possible) mainstream adoption:
- Technology Trigger. The technology breakthrough that kicks off the cycle.
- Peak of Inflated Experience. Lots of publicity about how the breakthrough fundamentally changes the way we do something with a number of early success and failure stories.
- Trough of Disillusionment. The phase where you go through the “this is much harder than we thought and the early stage vendors are not delivering on what they promised.” This is usually followed by a vendor shakeout—new solution providers entering the field offering more features and functionality and entrenched ones retooling their offerings to satisfy early adopters and get ready for mass adoption.
- Slope of Enlightenment. More data is available on how the technology can benefit business, more adopters, more successes (rather than failures), more companies funding projects, and second and third generation solutions appear.
- Plateau of Productivity. Mainstream adoption begins.
This year, Gartner’s Hype Cycle included (for the first time) “big data,” including it with among others, gamification, the Internet of Things, and Image Recognition, trending up on the Peak of Inflated Experience. Far more interesting, is that of these items mentioned, big data is two to five years out from mainstream adoption while the others are five to ten years out.
As a big data player, we (as in all of us at PatternBuilders) agree with this assessment. Certainly, from a prospect and partnership perspective, we are seeing much more activity, much of it unsolicited (which is great when you are a small privately held company because it’s a validation that your product—in our case, the Analytics Platform—works as advertised and that less of your budget is spent on generating interest in your solution).
But when I look at the big data industry, I am not sure that we are on the upwards slope of the Peak of Inflated Experience (I have just got to say that Gartner’s dramatic word choices for these stages always gives me the giggles but I digress). I believe that we are actually on the downward slope moving into the Trough of Disappointment where the first generation providers, like Hadoop and CloudEra, have already suffered the slings and arrows of some successes mixed with failures and second generation providers, like us, are entering into the industry.
In the case of Hadoop and CloudEra, I am talking about high end (in terms of dollars due to resource and infrastructure costs) big data solutions directed at the Fortune 50 that can only perform batch mode analytics. In our case, I am talking about four deployment models that are designed to meet the needs and requirements of different size companies and budgets, a development platform that requires much less resource-intensive work, streaming analytics which enables real-time or batch analytics, and exploratory analysis which allows you to find out what you didn’t know you knew. In other words, the second generation vendors have expanded upon what the first generation offered, looking again at technology and saying, “We can do this better.”
In my mind this perfectly positions the industry as moving through the Trough of Disillusionment and moving up towards the Slope of Enlightenment (come on—can you image saying these phase names with a straight face?) where more companies are seeing the benefits of big data analytics and starting to engage in projects around it. Although I am not a research analyst, I would point to a number of studies that I have blogged about previously—all of which point to this trend:
- McKinsey’s in depth analysis of the potential of big data across five key areas.
- The MIT Sloan Management Review study that looks at how many companies use big data as a key competitive differentiator.
- IDC’s paper on “Extracting Value from Chaos.”
So, while I am happy to see that big data made it on to Gartner’s Hype Cycle for 2011, with all due respect to Gartner, I think that the industry is already experiencing vendor shakeout and the second generation big data solution providers are about to show everyone just what big data can do!
Entry filed under: big data, Data, General Analytics, Technology. Tags: analytics, big data, exploratory analysis, Gartner, Hadoop, hype cycle, PatternBuilders Analytic Framework, streaming analytics, Technology.