Big Data Big Analytics

Getting Value From Your Data Series: The Road May Be Rocky But It’s Well Worth the Effort!

September 30, 2014 at 4:06 pm Marilyn Craig 4 comments

By Mary Ludloff and Marilyn Craig

Unless you’ve been asleep for the past couple of years, you, like us, have heard this phrase again and again:  Data is the new oil.  It certainly sounds great but what exactly does it mean? Here’s our take: Getting the most value out of your data can make you better at what you do as well as enable you to do more with what you have. In other words, there’s unrealized value in those data silos that all companies have. But make no mistake: the road to realizing data value is paved with good intentions and often times, poor execution and results.

Today, most companies are drowning in data—there’s historical data from operations, data from public sources, data from partners and acquisitions, data you can purchase from data brokers, etc.  These companies have read all the research and want to leverage their data assets to make “better” operational decisions, to offer their existing customer base more insights, to pursue new revenue opportunities. Of course, the real value in that data is derived from the business analytics that deliver the insights that drive better decisions. As we’ve said quite often on this blog: Data, without the proper use of analytics, is meaningless. If data is the new oil, think of analytics as the oil drills—you need both to be successful.

Sounds easy, right? But as we all know, there have been a lot of wrecks scattered along the analytics highway. Since we’ve worked on many big data analytics projects, we thought it might be useful to walk you through our methodology on what it takes—from inception to proof of concept to implementation and deployment—to navigate project pitfalls. To that end, we are going to use an “almost” real-world example inspired by some organizations we’ve worked with: Meet our parking meter company, fictionally named Acme. While Acme may not reflect the sexiest business model, it represents how most companies start spinning up analytics projects. While we won’t (and really can’t due to NDAs, etc.) reveal the organizations that inspired this example, we will give you lots to think about.  Our goal in this series of posts is to introduce you to our analytics project methodology using Acme as a case study.

Meet Acme Corporation—the King of Smart Parking Meters

Acme (a manufacturer of “smart,” aka IP-enabled, parking meters) Corporation’s customer service organization recently shared a number of questions they received from customers. The questions had a recurring theme—more insight into what is happening at and around customers’ traffic meters. For example:

• Who’s using the parking meters?
• Where are the most trafficked parking meters located?
• Could you infer what else people may be doing in the area by looking at location data? Are there stores and restaurants nearby?

Based on this feedback, Acme’s product managers believe there might be an opportunity to develop analytics that use the smart meter data.  If successful, the analytics could be bundled and offered up as a new product or feature, an additional revenue stream for the company. The product management team starts to scope out what this analytics project might entail.

Now, this shouldn’t surprise regular readers of our blog, but an analytics project should be treated in many respects like any other large scale project implementation. You need to think through what’s important (your objectives), what you have, what you need, and who you want to work on this new opportunity.  Check out our ongoing series on “Big Data Projects” for more on this topic or our Analytics Audit for an overview of our methodology.

So what will it take for Acme to satisfy its customers and extract value from the smart meter data? Well, since they are essentially creating a product that is based on the analysis of data, the steps they use in their standard product creation or development process apply here as well. But in addition to those fundamentals, there are a series of other questions and considerations that need to be addressed in order to develop the analytics and ultimately, extract value from them.  The preceding table slices an analytics effort into four key categories:

• Customer—Your targeted audience. Now obviously there’s a customer but you need to be able to understand their specific needs and business pain points as well as how your analytic results meet them. After all, I am sure that we have all been involved with projects that produced great analysis but there was only one problem: no one cared.
• People—No surprise here as people are your most valuable development asset (see our series on the data science team). But the talent that makes up that team should go far beyond the data scientist, business analyst, most seasoned (and smart) developer, and someone from IT. You need to involve the people that collect, store, use, and analyze the data. Also, you will want to include some customers that reflect your targeted need.
• Data—What you base your analytics on. It’s important to understand the data sources you need for your project as well as the quality of each data source. Each source needs to be assessed for accuracy and completeness (it’s a given that all data is dirty to some extent), scalability and sustainability, and most importantly, ownership, privacy and security This is especially true if you are going to “mashup” and analyze your internal data with public sources or data from other sources. For example, a partner may share their data (possibly also from multiple sources) with you.
• Technology—Yes, this is a very large, complex, and complicated topic with lots of moving parts (Hadoop? NoSQL? Cloud? HTML5?) which is why we built a platform that takes care of most of it. But you do need to understand the technology requirements and the tradeoffs you may need to make (like batched data analysis versus streaming data analysis) when picking your analytics platform.

Acme’s Data Collection and Possible Usage

Acme Corporation makes and sells its parking meters to municipalities globally.  The meters themselves might be old school coin-operated but they are more likely to be the new-fangled IP-enabled, credit card-ready meter (making life so much easier for those of us without the proper coins!).  Now, there is a high volume of data produced by each parking transaction (including a ton that falls into the category of PII data which has privacy implications that we will discuss in a later post) that each and every one of Acme’s meters produces every day, all day long, in cities like our own, San Mateo, or Seattle, Denver, London, Amsterdam, etc.  Acme’s business and product development folks want to take the meters’ transactional data (a gold mine for add-on revenue) and figure out the ways in which they can use analytics to derive more value for themselves and their customers.  Furthermore, the analytics their customers need may require augmentation of the smart meter data with public and purchased sources like weather and freeway traffic sensor data.  In order to start the process, Acme must determine who its customer is, what they want, and how they will benefit from the data (addressing the issues in the Customer category).

Now that we’ve found out more about what Acme does, the next series of posts will drill down into the four key categories. First up: Identifying your target analytics customer.

Entry filed under: Big Data Project, General Analytics. Tags: analytics, fast data, streaming analytics.