Posts filed under ‘big data’

In a pii (Privacy, Identity, Innovation) Conference State of Mind

By Mary Ludloff

pii2014Although this year has been extremely busy for us, Terence and I always find time for this event: The Privacy Identity Innovation Conference.  Natalie Fonseca, the Co-Founder and Executive Producer of it, is the driving force behind its ongoing success. This year’s program focuses on:

“… the latest developments in areas like mobile, biometrics, the Internet of Things and big data. Learn about emerging trends and business models driving the personal information economy, and get guidance on developing strategies and best practices to build trust with your users.” (more…)

November 9, 2014 at 1:33 pm Leave a comment

A Sneak Peek at Our New HTML 5 UI and Geek Love for Some of the Libraries Used in Building AnalyticsPBI4Azure

By Terence Craig

AnalyticsPBI Coming SoonDrumroll please! After nearly a year of development work, we are about to offer early access to the first real-time/streaming analytics solution software appliance for the cloud – AnalyticsPBI for Azure.  There will be more forthcoming on the product launch but the new UI is so cool I had to show it off a bit.

We will be following up with a formal launch and Early Access Program (EAP) signups in the next couple of weeks so watch this space and patternbuilders.com for details – the big data analytics market is about to change in a big way! Here’s a sneak peek on what we’ve been working on.

For the geek part of my blog I am going to give a shout out to three libraries that we are using – all have made a huge difference in the product’s performance, scalability, and usability. The first two libraries come from Microsoft – Reactive Extensions and TPL Dataflow.  The third library is the open source math and statistics library, Math.Net.

(more…)

November 8, 2013 at 2:07 pm 2 comments

pii2013: Building Trust in the Data Driven Economy—Hope to see you there!

By Terence Craig

pii2013As entrepreneurs at a growing startup there are very few things that are exciting enough to divert even a tiny bit of our attention from giving our customers the world’s best streaming analytics technology.  And while my co-founder Mary and I have been known to disagree on what those things might be, we are always in agreement that the Privacy Identity Innovation Conferences (pii) are the best conferences for bringing together leading voices from technology, science, and government for the critical discussion(s) of what Privacy and Identity mean in the age of the NSA, Facebook, and Internet  of things. pii2013 is being held in Seattle this year to (as their website states):

“Explore emerging technologies and business models, and highlight strategies and best practices for building trust with users. From news reports of increasing government surveillance to stories about startups using customer data in ‘surprising’ ways, there’s no shortage of examples illustrating why now is an important time to talk about innovation and trust. It’s a critical conversation about the future of privacy, identity and reputation that you won’t want to miss.” (more…)

August 12, 2013 at 2:18 pm Leave a comment

Strata West, Law, Ethics, and Open Data: Smart People Solving Some Very Hard Problems

By Terence Craig

Strata 3Last week the Bay Area was treated to another great Strata West hosted by the O’Reilly team. For those of you who weren’t able to make it, keep checking strataconf.com for updates on the videos and speaker slides—one of the great things about this conference is that many of the sessions are available to anyone as are the videos and slides.

I had the pleasure of co-hosting the Law, Ethics, and Open Data track with my friend and fellow O’Reilly Author (and Civilization devotee), Alex Howard.  Alex is O’Reilly’s government reporter and his book, Data for the Public Good, is a must read. Our track was two days long and featured thoughtful sessions and speakers–bringing together people who are solving difficult technology problems and then showing us how those problems and solutions are impacting lives and society. If you check out my tweets from last week you’ll see my 140 character attempts to highlight some of the sessions.  Here is a “longer” version of the highlights of the sessions I hosted:

  • Fred Trotter and DocGraphFred actually tweeted his presentation as he was giving it, so check out @fredtrotter for last Thursday starting around 10:40 am PST.  A presentation of 140 character sound bites made for a very succinct message.  He’s done some amazing work creating the DocGraph, probably the largest public social graph in the world, showing the referral relationships between doctors in the US. You can view a nice visualization his team has done here. (more…)

March 8, 2013 at 6:02 pm 1 comment

A Big Data Showdown: How many V’s do we really need? Three!

By Mary Ludloff

3 vs of big dataMarilyn Craig (Managing Director of Insight Voices, frequent guest blogger, marketing colleague, and analytics guru) and I have been watching the big data “V” pile-on with a bit of bemusement lately. We started with the classic 3 V’s, codified by Doug Laney, a META Group and now Gartner analyst, in early 2001 (yes, that’s correct, 2001). Doug puts it this way:

“In the late 1990s, while a META Group analyst (Note: META is now part of Gartner), it was becoming evident that our clients increasingly were encumbered by their data assets.  While many pundits were talking about, many clients were lamenting, and many vendors were seizing the opportunity of these fast-growing data stores, I also realized that something else was going on. Sea changes in the speed at which data was flowing mainly due to electronic commerce, along with the increasing breadth of data sources, structures and formats due to the post Y2K-ERP application boom were as or more challenging to data management teams than was the increasing quantity of data.”

Doug worked with clients on these issues as well as spoke about them at industry conferences. He then wrote a research note (February 2001) entitled “3-D Data Management: Controlling Data Volume, Velocity and Variety” which is available in its entirety here (pdf too). (more…)

January 17, 2013 at 7:06 pm 4 comments

“Hadoopla”

© Marqin Cook

By Terence Craig

I had to miss Strata due to a family emergency. While Mary picked up the slack for me at our privacy session, and by all reports did her usual outstanding job, I also had to cancel a Tuesday night Strata session sponsored by 10Gen on how PatternBuilders has used Mongo and Azure to create a next generation big data analytics system.   The good news is that I should have some time to catch up on my writing this week so look for a version of what would have been my 10Gen talk shortly. In the meantime, to get me back in the groove, here is a very short post inspired by a Forbes post written by Dan Everett of SAP on “Hadoopla”

As a CEO of a real-time big data analytics company that occasionally competes with parts of the Hadoop ecosystem, I may have some biases (you think?).  But I certainly agree that there is too much Hadoopla (a great term).  If our goal as an industry is to move Big Data out of the lab and into mainstream use by anyone other than the companies that thrive on and have the staff to support high maintenance and very high skill technologies, Hadoop is not the answer – it has too many moving parts and is simply too complex.

To quote from a blog post I wrote a year ago:

“Hadoop is a nifty technology that offers one of the best distributed batch processing frameworks available, although there are other very good ones that don’t get nearly as much press, including Condor and Globus.  All of these systems fit broadly into the High Performance, Parallel, or Grid computing categories and all have been or are currently used to perform analytics on large data sets (as well as other types of problems that can benefit from bringing the power of multiple computers to bear on a problem). The SETI project is probably the most well know (and IMHO, the coolest) application of these technologies outside of that little company in Mountain View indexing the Internet. But just because a system can be used for analytics doesn’t make it an analytics system…..

Why is the industry so focused on Hadoop? Given the huge amount of venture capital that has been poured into various members of the Hadoop eco-system and that eco-system’s failure to find a breakout business model that isn’t hampered by Hadoop’s intrinsic complexity, there is ample incentive for a lot of very savvy folks to attempt to market around these limitations.  But no amount of marketing can change the fact that Hadoop is a tool for companies with elite programmers and top of the line computing infrastructures. And in that niche, it excels.  But it was not designed, and in my opinion will never see, broad adoption outside of that niche despite the seeming endless growth of Hadoopla.

October 24, 2012 at 1:39 pm 1 comment

Big Data and Science: Focus on the Business and Team, Not the Data (Part 3 of 3)

By Mary Ludloff

Let me tell you a little secret: I always know when I am talking (and working) with a company that has successfully launched big data initiatives. There are three characteristics that these companies share:

  1. A C-level executive runs the “[big] data operations.”
  2. The Chief Data Officer (even if they are the CIO) has a heavy business/operations background.
  3. The data team is focused on the “business,” not the data.

Did you notice that technology and data science are not reflected in any of the characteristics? Some of you may consider this sacrilege—after all, we are operating in a world where technology (and I happily work for one of those companies) has changed the data collection, usage, and analysis game. Colleges and universities are now offering master degrees in analytics. The role of the data scientist has been pretty much deified (I refer you to Part 1 of this series). And we all need to be very worried about the “talent shortage” and our ability to recruit the “right analytical team” (I refer you to Part 2 of this series).

Yes—technology has had a tremendous impact on how much data we can collect and the ways in which we can analyze it but not everyone needs to be a senior computer programmer. Yes—we all should strive to be more mathematically inclined but not all of us need Master’s or PhD’s in statistics or analytics. Yes—some companies, based on their business models, may have a staff of data scientists but others may get along just fine without one (with the occasional analytics consultant lending a hand). (more…)

October 20, 2012 at 4:50 am 4 comments

Older Posts


Video: Big Data Made Easy

PatternBuilders Corporate

Special privacy section!

Previous Posts


%d bloggers like this: