Welcome back to the second post in our series on how to get value from your data. As we stated in a previous post:
“Data, without the proper use of analytics, is meaningless. If data is the new oil, think of analytics as the oil drills—you need both to be successful.”
Of course, getting to “success” is not easy as anyone involved in an analytics project will tell you. This series walks you through our methodology on what it takes—from inception to proof of concept to implementation and deployment—to navigate project pitfalls. Now most of us have been involved with great analytics projects that answered no real need. In this post, we take a look at the customer, their pain points, and what benefits they may derive from your analytics project. In other words: Who is your target customer? (more…)
Unless you’ve been asleep for the past couple of years, you, like us, have heard this phrase again and again: Data is the new oil. It certainly sounds great but what exactly does it mean? Here’s our take: Getting the most value out of your data can make you better at what you do as well as enable you to do more with what you have. In other words, there’s unrealized value in those data silos that all companies have. But make no mistake: the road to realizing data value is paved with good intentions and often times, poor execution and results.
Today, most companies are drowning in data—there’s historical data from operations, data from public sources, data from partners and acquisitions, data you can purchase from data brokers, etc. These companies have read all the research and want to leverage their data assets to make “better” operational decisions, to offer their existing customer base more insights, to pursue new revenue opportunities. Of course, the real value in that data is derived from the business analytics that deliver the insights that drive better decisions. As we’ve said quite often on this blog: Data, without the proper use of analytics, is meaningless. If data is the new oil, think of analytics as the oil drills—you need both to be successful. (more…)
We read an interesting paper and post about Google Flu Trends (GFT) and its foibles last week. The paper points out a couple of lessons that those of us living in the big data analytics world have learned the hard way but the dangers are worth revisiting as tools like ours (AnalyticsPBI for Azure) begin to move big data analytics into the mainstream of organizational practices. After all, our tool (and others like it) makes it easy and even fun for analytics junkies to use all those available zettabytes of data and answer questions that they’ve long wondered about. But the paper also reminded us of the dangers of ignoring the natural cycles of an analytics process that we talked about in this recent post. If Google followed the PatternBuilders Analytics Methodology, they might have avoided many of the errors that GFT is now spitting out. In fact, the authors of the paper point out that:
“Although not widely reported until 2013, the new GFT has been persistently overestimating flu prevalence for a much longer time. GFT also missed by a very large margin in the 2011-2012 flu season and has missed high for 100 out of 108 weeks starting with August 2011… This pattern means that GFT overlooks considerable information that could be extracted by traditional statistical methods.”
This overestimation is attributed to two primary factors: data hubris and algorithm dynamics. (more…)
A recent conversation with a client reminded me that no matter how crazy and exciting the Big Data world gets, it is still critical to understand what your goals are and where you are in the process of reaching those goals. Having a good foundation in “what’s important” is critical before you jump into the wild world of Big Analytics.
For example, in big data (well, actually all data but I digress) “Reporting” and “Analytics” are very different functions. But I often find our customers and prospects grappling with how to distinguish one from the other and as a result, confusing reporting with analysis and losing track of their real goals.
New Year (2014) Rumination: Death of privacy as we know it? Or inflection point signaling better things to come?
I am, and always have been, a glass half-full kind of gal. In fact, way back in September 2011 when Terence and I published our book on Privacy and Big Data, I was far more optimistic than he was on the future of privacy—of course, it’s easy to sound optimistic when your co-author states that privacy is dead. (And yes, we are still working on our book update but we do have day jobs and a significant release in the works so it is slow going but going it is.)
At that time, those in the “digital privacy know” characterized our book as a decent overview. Our intent at the time was to help those NOT in the “digital privacy know” get their arms around the privacy issues from a legislative, corporate, and government perspective. To our surprise, those not in the know included lots of folks in the high tech community! We did a number of interviews and dealt with informed and somewhat uninformed media folk—those in the mainstream focused on social media and those on the fringes (left and right) wanted to do deep dives into legal issues, government uses of data, and fourth amendment rights. Some seemed to think that we were members of the tin foil hat brigade, others that we were naïve, and still others that we were on point. (more…)
This past weekend the Philippines and surrounding areas were decimated by super typhoon Haiyan. The storm was estimated to have sustaining winds of 195 mph with winds gusts up to 235 mph. It is believed to be one of the strongest storms ever recorded.
For those of us who have lived through typhoons and hurricanes (myself included) and felt the power of those storms, what we saw was nothing compared to what the Philippines endured. The death toll is now estimated to be above 10,000 and climbing and the devastation it left is likened to the destruction wrought by the 2004 Indian Ocean tsunami.
It will take years to rebuild and recover from such a disaster and our hearts go out to the people impacted by the typhoon as well as the relief workers who are providing much needed aid to that region. For those of you who would like to help, the New York Times and Reuters have provided lists of organizations (with links) that are providing aid. I would also point you to a local organization (in the Washington area), World Vision, that is intent on providing relief to children and families devastated by the typhoon.
In times like these we are all reminded of the fragility of life, the power of nature, and the undeniable fact that we are truly all in this together. Please help if you can.
A Sneak Peek at Our New HTML 5 UI and Geek Love for Some of the Libraries Used in Building AnalyticsPBI4Azure
Drumroll please! After nearly a year of development work, we are about to offer early access to the first real-time/streaming analytics solution software appliance for the cloud – AnalyticsPBI for Azure. There will be more forthcoming on the product launch but the new UI is so cool I had to show it off a bit.
We will be following up with a formal launch and Early Access Program (EAP) signups in the next couple of weeks so watch this space and patternbuilders.com for details – the big data analytics market is about to change in a big way! Here’s a sneak peek on what we’ve been working on.
For the geek part of my blog I am going to give a shout out to three libraries that we are using – all have made a huge difference in the product’s performance, scalability, and usability. The first two libraries come from Microsoft – Reactive Extensions and TPL Dataflow. The third library is the open source math and statistics library, Math.Net.