Archive for August, 2013

Announcing the Big Data Hawkathon Winners

What happens when you unleash a few dozen geeks on some big data with a powerful analytics platform like HP Vertica for a day?

There was no fowl play and the results were starling. And yes, they were winging it… Sorry, I couldn’t resist.

At the HP Vertica Big Data Conference, HP Vertica teamed up with the Audubon Society to sponsor the first-ever Vertica Big Data Hackathon — or, as it affectionately became known, Hawkathon! At this event, teams of one to five people spent the day using the HP Vertica Analytics Platform to analyze/visualize interesting Big Data sets. Each team had about seven hours to build something cool. The teams with the best solutions were awarded prizes.

The goal of the Hawkathon was to give people an opportunity to explore HP Vertica’s capabilities in a creative, non-work environment. There were five data sets pre-loaded on the cloud-hosted HP Vertica clusters, including:

  • Apache public email archives
  • Wikipedia website traffic statistics
  • Global weather observations

Perhaps the most interesting data set was the Audubon Christmas Bird Count (CBC) database. The CBC, which provides critical data on bird population trends, was most frequently chosen data set by the Hawkathon teams.
There were really no rules for the Hawkathon. Teams had full root access and could use any other tools they wish. At the end of the day, each team had to demonstrate their work to all other participants and a team of judges. Judging considered the following five categories:

  1. Technical Merit/Use of HP Vertica Features
  2. Completed Solution/Level of Functionality
  3. Ease of Use/Visualization
  4. Impactful/Solve Real-World Problem
  5. Creativity/Wow Factor

After demonstrations, the judges conferred and chose the top three teams. Prizes included HP Slate tablets, Beats Audio headphones, and Amazon gift cards.

The Compete team (the digital marketing optimization service) took third place. Led by Principal Architect, Seth Madison, the Compete team custom developed an application that displayed bird species population trends over time (see graphic above).

The one-man-team from Looker, the business intelligence software company, took second place. Lloyd Tabb, Looker founder and CTO, implemented their self-service BI tool to make CBC data exploration easy.

The first place team did an impressive analysis on the effect of weather on the quality of wine. This team had members from Deloitte Consulting (Jason Chiu), HP Global Delivery (Filip Nowicki), and HP Labs (Daniel Gmach and Alvin AuYoung). They combined several technologies and data sets to isolate the atmospheric attributes that impact wine production and even went so far as to make some wine-type recommendations. Though there were no sommeliers in the room that could confirm or dispute their findings, it is a testament to the infinite uses of HP Vertica and Big Data in general.

This Hawkathon was very successful, in my opinion. The room remained highly energized and participants really got to use the HP Vertica Analytics Platform in new and interesting ways that they might not have had time to do back in the office. Many teams did not want to stop working!

“We were very excited by the degree of interest in Audubon’s Christmas Bird Count dataset, the largest and longest running animal census on the planet. We saw considerable potential in the ideas and applications developed by the competing teams. The results of the Hawkathon offer strong support for our efforts to collaborate with new partners—such as HP Vertica—to address critical conservation challenges, both present and future, ” says Candan Soykan, Quantitative Ecologist at Audubon.

The Hawkathon provided a venue for exploring the capabilities of the HP Vertica Analytics Platform to work with massive quantities of data. Given the explosion of data we’re witnessing, it is more important than ever to have powerful tools to operate upon and monetize this data. As such, we’relooking forward to holding additional hackathons in the near future. Interested in hosting a hackathon or joining one at HP Vertica headquarters in Cambridge? Just let us know!

Doing Everything with Big Data

Earlier this year we invited Chris Wegryzyn, Director of Data Architecture for the Democratic National Committee, to participate in “The Disruptive Power of Big Data,” a webinar discussing how the Vertica analytics platform played a central role in achieving his organizations key focus—to measure everything. As this was one of the most successful programs we’ve run recently, it was a true pleasure to have him back speaking at the HP Vertica Big Data Conference 2013.

The story of how the DNC integrated data analytics into their organization is incredible. They went from having no Big Data program in 2008 to creating one of the most impressive big data analytics organizations we’ve seen in 2012—And Vertica was the heart of their platform. Vertica enabled them to build a single, comprehensive platform that empowered staffers throughout the campaign to utilize data, everything from registered voter lists and social data to data collected by their army of volunteers, to make the right decisions in communicating their message to the voters.

At Vertica we are always talking about the power of real-time insights and how timely business intelligence is critical to making the smartest choices, but the DNC had to take this a step further. As Chris mentioned during the conference, his organization had “the mother of all deadlines.” Every decision had to matter, from each email sent out to potential voters to what media purchases they should make in any given market. All of their data had to be working for them.

Check out this video where Chris discusses how they achieved this. For his organization Big Data analytics became rooted in their philosophy allowing them to translate all of their raw data into insights that fueled their success. At one point in the video, responding to a question about the role of data, Chris smiled and said, “We did everything with data.” That’s the kind of message we can get behind at Vertica.

Did you miss the HP Vertica Big Data Conference? Check out our conference wrap up page to see some of the highlights. We’ll continue to post updates to the page in the coming weeks so don’t be a stranger.

Bird Watching, Vertica, and a Room Full of Data Geeks

Each December the National Audubon Society coordinates the Christmas Bird Count, when masses of avid birdwatchers dust off their binoculars and set out to record the number of birds seen in a predetermined area. This centuries-old survey, first proposed to end the more gruesome tradition of Christmas “side hunts” (when the birds were shot on sight instead of counted), draws thousands of participants from around the globe to collect and record regional population data of birds. This rich data set is used in conservation data each year, but what happens when you let a room full of data geeks loose on this information?

That’s exactly what we set out to answer by hosting our first hackathon (also known as the HAWKathon) at the HP Vertica Big Data Conference 2013. Multiple teams combed through a number of rich data sets, including the Audubon data, Apache data, and Wikimedia and Wikinews page counts, to find hidden data gems and compete for the best discoveries.

Our second place winner, Lloyd Tabb, Founder, Chairman and CTO of Looker, did just that. Lloyd built his solution using the Vertica analytics platform and Looker, his query-based BI tool, to visualize the data. While the dashboard shows a staggering amount of data, one of my favorite observations noted was that three regions, California, Texas, and New York, counted more birds than there were species. I didn’t know Texas had so many bird lovers. Take a look at this graphic to see some of the other discoveries Lloyd made.

This year’s hackathon was an overwhelming success and a huge thanks to all of the participants and organizers, especially the Audubon Society for providing the data and making the hackathon such a great event. We’ll see you at next year’s HP Vertica Big Data Conference.

Stay tuned for more posts about the hackathon and Big Data Conference in the coming weeks.

Get Started With Vertica Today

Subscribe to Vertica