The Big Data Hammer?

I've got my hammer, where is your nail?

With all the tech-media hype around big data I am reminded of the proverb ...

if all you have is a hammer, everything looks like a nail

Basically the proverb is an attempt to sum up the human condition of being single-minded and biased toward your own expertise.  This is where I fear we are headed with the "big data" explosion.

Insert bold assertion here

I am of the opinion that ...

 the software and consulting industry is viewing "big data" as the new technology hammer and all they can see are nails

Don't get me wrong, I see the valid use cases for a new way of doing things.  I am also as excited about this as everyone else seems to be.  However, I think there are some aggressive claims out there that scare me on behalf of those who are charged with managing data.

Here is an example ...

[gallery id="2483"]

[ref] Big data: The next frontier for innovation, competition, and productivity, May 2011 | by James Manyika, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Angela Hung Byers, http://www.mckinsey.com/insights/mgi/research/technology_and_innovation/big_data_the_next_frontier_for_innovation [/ref]

Dwarfed, perhaps intentionally, by the larger, more attention grabbing graphics was the keyword could in every one of those claims.  And at the bottom of the graphic was another favorite of mine ...

There seems just a few issues with capitalizing on "big data".  Things like talent, which is in extremely low supply, and the legality of using inaccurate data with negative consequences (Inaccurate data?  what?  <--sarcasm)

Don't just take my random interpretation.  Here is a great deck on when to and, more importantly, when not to use a NoSQL (aka "big data") solution:

[slideshare id=10487652&doc=gotoslides-mailable-111206135716-phpapp01]

There are some great slides towards the end of the presentation (slide 31+) that cover when and when not to use Riak, a NoSQL database.  I think this quote sums up my main point ...

Don't use Riak ... if you don't have problem right now

This is a great point, so let's repeat it!

If you don't have a problem right now with your current SQL based solution, don't throw "big data" technologies at it.  [ref] Perhaps a more important and subtle point in that statement is to know what problem you need to solve before you jump into the "big data" deep end. [/ref]

In other words

Make sure you need to hit a nail before you swing a hammer!

Sound off!

[polldaddy poll=6598510]