More than 3,000 attendees converged on the sold-out O’Reilly Strata Conference and Hadoop World 2012 in New York City to gain some clarity on arguably the biggest high-tech megatrend in recent years: Big Data.
From a 100,000-foot view, the majority of attendees—from press to developers to exhibitors to event staff—understood that we are generating a nearly incomprehensible amount of data, really Big Data. And there’s no reason to believe that this Big Data will continue to grow by orders of magnitude, given the proliferation of:
- Social media
- Connected cars, household appliances, medical equipment, and other machines
- Telco and financial data
- Medical records
- And more
But from my conversations, attendees came to the show to understand how their organization could manage, analyze, and ultimately monetize this Big Data, and, specifically, how Hadoop could help with that effort.
As a newbie to this space, I could relate to the quizzical faces of attendees, barraged with messages claims as the next Big Data solution, but with very different offerings—everything from search engines to hosted solutions to ETL tools to even staffing resources.
Hadoop in itself comprises a uniquely named set of technologies: Hive, Sqoop, Pig, Flume, etc. Despite the unusual terminology, the Hadoop-focused sessions proved educational and featured an impressive range of real-world case studies even large companies (such as Facebook) using Hadoop to store and analyze an impressive amount of Big Data.
But the question still remains: is Hadoop the answer or are there other technologies that can either complement or serve as a better path?
As is often the case when choosing technology, the answer is “It depends on your business need.”
At HP, many of our customers used Hadoop for batch processing before ultimately adopting the HP Vertica Data Analytics Platform to manage and analyze their Big Data for sub-second query response times.
Other customers, particularly with the Hadoop Connector released with HP Vertica Version 6, use the technologies together to seamlessly move data back and forth between Hadoop and HP Vertica.
Which use cases do you feel are a good fit for Hadoop and how can we provide better integration with our platform? Let us know.
We’re passionate about providing the data analytics platform to help you obtain answers from your Big Data questions and add some clarity, in the process.