Posts tagged ‘Hadoop’

Big Data and Cloud not a fit? Comments on Infoworld Article

By Terence Craig

Since Disqus seems to have completely eaten (bleh) my comment on @davidlinthicum’s very interesting InfoWorld post – Big data and the cloud: A far from perfect fit, I decided to just expand my comments and make a short blog post out of it. IMHO the problems that David is describing are more a reflection of problems with batch oriented technologies like Hadoop (more on my take on Hadoop here) in the cloud than a general problem for cloud based big data solutions.

Computing always has, and probably always will have, a bias towards creating batch focused technologies at the beginning of any large paradigm shift.   But as new technologies are absorbed, understood, and move from early adopter to more mainstream use, the batch paradigm will inevitably start to shift to streaming and real-time. We have seen this again and again (from punch cards to touch sensitive tablets, downloaded media to streaming media, DOM to SAX parsers, HTML to Ajax, paper maps to real-time GPS). The reason this evolution almost always occurs is simple: humans live and think in real-time and when our tools do as well we are more productive and happier.  So why do we have this bias for batch processing in our first generation computational technologies? Simply put, because batch processing is a lot easier.

(more…)

February 23, 2012 at 3:01 pm Leave a comment

FinancePBI Begins its Shakeout Flight in the Cloud

By Terence Craig

I have been a little quiet on the blogging front recently as I and the rest of the PatternBuilders team have been focused on getting ready to launch our new financial services application: FinancePBI. It is the first cloud-based analytical platform for the Financial Services market.  While this is our first public announcement of our entry into the market, behind the scenes we have been gearing the company up for a big splash for several months:

  • Partnered with ActiveFinancial one of the premier real-time stock ticker vendors in the world.  Look for more data partnerships shortly.
  • We have added Doug Jeffrey to our board of advisors and board of directors.  Doug is an executive with deep Wall Street and startup expertise who has already done outstanding things in the short time he has been with us.
  • We have also partnered with the University of Sydney to use our technology to examine the influence of primary sources (NY Times, etc.) and secondary social media (Twitter, etc.) content on a company’s stock price over a 12 month period. This project will be done exclusively in the cloud and it’s our hope is that we will be able to convince our commercial partners to allow this PatternBuilders instance to be available to the general public. Of course, this would happen after the research is published. (more…)

February 7, 2012 at 8:04 pm 1 comment

No, Hadoop Doesn’t Own Big Data Analytics!

By Terence Craig

A number of folks have asked me if I was concerned about Microsoft’s  recent announcement that they would be partnering with HortonWorks and abandoning their own distributed processing technology for Hadoop.  While I thought this was an unfortunate choice on Microsoft’s part (the Dryad project’s implementation of multi-server Linq was pretty compelling), since HPC is a small part of Microsoft’s business, it probably made sense from a business standpoint.   In any case, we (as in all of us at PatternBuilders) are not concerned and just to be clear: we don’t believe that this announcement (or any other) means that the many Hadoop ecosystem players own the still forming big data analytics market.

That is not to say that the announcement isn’t proof of the strength of the Hadoop ecosystem. Hadoop is a nifty technology that offers one of the best distributed batch processing frameworks available, although there are other very good ones that don’t get nearly as much press, including Condor and Globus.  All of these systems fit broadly into the High Performance, Parallel, or Grid computing categories and all have been or are currently used to perform analytics on large data sets (as well as other types of problems that can benefit from bringing the power of multiple computers to bear on a problem). The SETI project is probably the most well know (and IMHO, the coolest) application of these technologies outside of that little company in Mountain View indexing the Internet. (more…)

December 12, 2011 at 1:41 pm 3 comments

Gartner, Hype Cycles, and Big Data

By Mary Ludloff

Gartner Hype Cycle Greetings all! While we’ve been super busy at PatternBuilders working on a destination application that we are all very excited about, doing some development work, talking with potential partners and prospects, AND not to mention the fact (but I will) that Terence and I are getting close to the finish line on our book, I came across this interesting article on Gartner’s hype cycle and the recent addition of big data to it.

Now, I am pretty sure that you all are familiar with Gartner’s magic quadrant methodology that essentially evaluates all the particular technology players in a specific industry across four quadrants: challengers, leaders, niche players, and visionaries. For those companies looking at vendors for a specific solution, the magic quadrant can help them understand how they stack up against each other. For the vendors, it’s an opportunity to take an objective look at the industry as a whole and understand what they do well and where they could be better. (more…)

August 17, 2011 at 8:37 am 2 comments

Big Data is the New Black… And it’s Here to Stay

By Terence Craig

Some impressions from O’Reilly’s Strata Conference 2011.

We had a great time at Strata 2011.  The O’Reilly folks did a great job of organizing a really well run show – hat tip to JB Wheatley for making life as easy as possible for speakers.

As a member of a company that has been carrying the big data, big analytics flag for a while, it was great to see a sold out show dedicated to the area that will provide one of the biggest challenges, if not the biggest challenge, for 21st century computing.   Our presentation seemed to go well – we had a good crowd and a bunch of folks came up to us during the rest of the week to discuss/argue some of the points we raised.  My statement about traditional databases/data warehouses not being up to the task definitely annoyed some people, but everybody that approached us was polite and thoughtful and we had some good discussions. (more…)

February 9, 2011 at 7:50 pm Leave a comment

Riding the Data Waterfall

Our new streaming analytics engine.

By Terence Craig

As promised, I am going to spend the next few posts discussing some of the new features in our analytics framework, otherwise known as PAF.  This is our largest and most complex release so far.  We are very proud of it—both in how far the framework has come and how closely it matches our vision of what a world class analytics system would look like when we started the company a few years ago.

One of my favorite features, and certainly the biggest change in this release, is that our analytics engine is now completely streaming based.  I think that this, along with our improved ad-hoc analysis support, is going to improve our customers’ day-to-day to experience with both calculating and using analytics in their businesses.    (more…)

January 26, 2011 at 6:01 pm 11 comments

Why isn’t BI a Solution for Big Data and Real-Time Analytics?

By Terence Craig

It’s a legitimate question that we get asked a lot: why can’t I use my multi-billion dollar BI system to manage my big data/real time analytics problem(s)?  I have found that my tongue in cheek answer “we need you buy our software and services because we have families to feed” though true, is not as compelling to customers as I would like which leads to today’s post.

To put the question another way: what can PatternBuilders and the rest of the new approaches to data and analytics like Hadoop or Mongo (which, by the way, our platform uses and is a great technology) offer you over and above what large BI company X offers? (more…)

December 20, 2010 at 11:56 am 4 comments

Newer Posts


Video: Big Data Made Easy

PatternBuilders Corporate

Special privacy section!

Previous Posts


%d bloggers like this: