Posts filed under ‘PatternBuilders Technology’

Big Data and Cloud not a fit? Comments on Infoworld Article

By Terence Craig

Since Disqus seems to have completely eaten (bleh) my comment on @davidlinthicum’s very interesting InfoWorld post – Big data and the cloud: A far from perfect fit, I decided to just expand my comments and make a short blog post out of it. IMHO the problems that David is describing are more a reflection of problems with batch oriented technologies like Hadoop (more on my take on Hadoop here) in the cloud than a general problem for cloud based big data solutions.

Computing always has, and probably always will have, a bias towards creating batch focused technologies at the beginning of any large paradigm shift.   But as new technologies are absorbed, understood, and move from early adopter to more mainstream use, the batch paradigm will inevitably start to shift to streaming and real-time. We have seen this again and again (from punch cards to touch sensitive tablets, downloaded media to streaming media, DOM to SAX parsers, HTML to Ajax, paper maps to real-time GPS). The reason this evolution almost always occurs is simple: humans live and think in real-time and when our tools do as well we are more productive and happier.  So why do we have this bias for batch processing in our first generation computational technologies? Simply put, because batch processing is a lot easier.

(more…)

February 23, 2012 at 3:01 pm Leave a comment

FinancePBI Begins its Shakeout Flight in the Cloud

By Terence Craig

I have been a little quiet on the blogging front recently as I and the rest of the PatternBuilders team have been focused on getting ready to launch our new financial services application: FinancePBI. It is the first cloud-based analytical platform for the Financial Services market.  While this is our first public announcement of our entry into the market, behind the scenes we have been gearing the company up for a big splash for several months:

  • Partnered with ActiveFinancial one of the premier real-time stock ticker vendors in the world.  Look for more data partnerships shortly.
  • We have added Doug Jeffrey to our board of advisors and board of directors.  Doug is an executive with deep Wall Street and startup expertise who has already done outstanding things in the short time he has been with us.
  • We have also partnered with the University of Sydney to use our technology to examine the influence of primary sources (NY Times, etc.) and secondary social media (Twitter, etc.) content on a company’s stock price over a 12 month period. This project will be done exclusively in the cloud and it’s our hope is that we will be able to convince our commercial partners to allow this PatternBuilders instance to be available to the general public. Of course, this would happen after the research is published. (more…)

February 7, 2012 at 8:04 pm 1 comment

No, Hadoop Doesn’t Own Big Data Analytics!

By Terence Craig

A number of folks have asked me if I was concerned about Microsoft’s  recent announcement that they would be partnering with HortonWorks and abandoning their own distributed processing technology for Hadoop.  While I thought this was an unfortunate choice on Microsoft’s part (the Dryad project’s implementation of multi-server Linq was pretty compelling), since HPC is a small part of Microsoft’s business, it probably made sense from a business standpoint.   In any case, we (as in all of us at PatternBuilders) are not concerned and just to be clear: we don’t believe that this announcement (or any other) means that the many Hadoop ecosystem players own the still forming big data analytics market.

That is not to say that the announcement isn’t proof of the strength of the Hadoop ecosystem. Hadoop is a nifty technology that offers one of the best distributed batch processing frameworks available, although there are other very good ones that don’t get nearly as much press, including Condor and Globus.  All of these systems fit broadly into the High Performance, Parallel, or Grid computing categories and all have been or are currently used to perform analytics on large data sets (as well as other types of problems that can benefit from bringing the power of multiple computers to bear on a problem). The SETI project is probably the most well know (and IMHO, the coolest) application of these technologies outside of that little company in Mountain View indexing the Internet. (more…)

December 12, 2011 at 1:41 pm 3 comments

Video: Big Data Made Easy. Sticky – see below for latest posts.

November 15, 2011 at 9:49 am 5 comments

All Together Now: All You Need is a Text Box!

By Terence Craig

All you need is text, Text is all you need (sing to the tune of The Beatles’ All you need is love).   If you are one of our regular readers you will remember that several months ago I wrote a manifesto on what the perfect analytics system would look like.  One of the last points was:

It must be as accessible as Excel (still the number one analytics tool in the world).

I was wrong – Excel is the number one non-specialized analytics tool in the world but in terms of usage, it is dwarfed in comparison to a very well know specialized analytics toolkit. The creators of this tool are a little company that you may have heard of:  it does no evil and analyzes the Internet to bring you back everything on the web based on a simple text query.  But behind that simple text box, Google has one of the most sophisticated analytics infrastructures in the world:

  • It can deduce your interests.
  • Give you the most relevant results.
  • And show you appropriate information based on them, as well as bring back highly personalized ads.

Google is not only the largest big data analytics company in the world, but it also has the easiest to use tools—proof that text is all you really need!

(more…)

October 14, 2011 at 3:22 pm 4 comments

No-SQL – Going All The Way

Going All The Way

We have recently made a big architectural change concerning our storage back-end and I wanted to talk about it.

Storage is key to any Big Data problem. As we’ve mentioned in prior posts, most of our performance bottlenecks and optimizations have to do with storage performance and architecture, as opposed to computation. Our architecture for the last few years has consisted of a hybrid approach with “no-SQL” analytics storage using MongoDB and “non-transactional” data stored in a traditional RDBMS, primarily SQL Server. There were a couple of reasons for this architecture. First, we started off entirely in RDBMS-land, because our initial design was done before no-SQL systems were really at a production-level of maturity. Second, most of our customers and prospects had traditional schemas and data organization – making integration easier if we could just use the same object model. (more…)

September 28, 2011 at 4:18 pm 1 comment

Maps: Lessons Learned

Recently we’ve been adding new user-friendly features to our platform and I’d like to talk about our map view. In particular, I want to discuss the lessons we learned from the map in the first version of the PAF (PatternBuilders Analytics Framework) versus the one in our new Silverlight client.

You may have already seen some screenshots of the map in our AJAX web client – when we released the first versions of PAF, we integrated with Google Maps to help users see their data on a map for quick comparisons and analysis. It’s always been a helpful tool, but suffered from a learning curve for new users and could potentially confuse people due to the way it displayed data.

The AJAX Client Map View

The AJAX Client Map View

Showing time series data on a map is a tricky proposition – the map is already two dimensional, and the addition of the two dimensions of time series analytics takes it into the 4th dimension. As exciting as it would be to see a four dimensional map view (we’d definitely be the only company doing it!), I don’t think most human beings would be able to understand it.

(more…)

September 6, 2011 at 8:04 am Leave a comment

Older Posts Newer Posts


Video: Big Data Made Easy

PatternBuilders Corporate

Special privacy section!

Previous Posts


%d bloggers like this: