Vertica

Archive for the ‘Data Scientists’ Category

A Sneak Peek of HP Vertica Pulse, Harnessing the Volume and Velocity of Social Media Data

The Web provides us with a myriad of ways to express opinion and interest—from social sites such as Twitter and Facebook to blogs and community forums to product reviews in ecommerce sites to many more. As a result, customers have significant influence in shaping the perceptions of brands and products. The challenge for the managers of those entities on which opinion and interest is expressed is to understand, in an automated way and in as close to real-time as possible, what people are talking about and how they feel about those topics, so that they can better understand and respond to their community.

HP Vertica Pulse — now in private beta — is HP’s scalable, in-database answer to the problem of harnessing the volume and velocity of social media data. Executed through a single line of SQL, HP Vertica Pulse enables you to extract “attributes,” or the aspects of a brand, product, service, or event that your users and customers are talking about; and the ability to assign a sentiment score for each of these attributes, so that you can track your community’s perception on the aspects of your business that your community cares about. Understand whether your customers are looking for a particular feature, how they react to a facet of your product or service as you anticipated, or if they are suddenly encountering problems. See how their perceptions change over time.

We used HP Vertica Pulse at HP Discover to capture attendee sentiment. We captured tweets related to HP Discover, the Tweeter’s screen name, and the timestamp. We ran the tweets through HP Vertica Pulse and visualized the results in Tableau. The whole effort was up and running in just a few hours. The screenshot below shows the major aspects:



 

  • We watched the interest of the crowd change over time. With each successive keynote, we saw new initiatives and people appear in the word cloud. Meg Whitman received a lot of press, as did HP’s New Style of IT and HAVEn. So did Kevin Bacon, who participated in Meg’s keynote.
  • HP Vertica Pulse surfaced news in the data analytics world. In a trial run using tweets related to data analytics, we saw “Walmart” — not a common name in the world of analytics — appear in the word cloud. A quick drilldown in Tableau revealed that Walmart recently purchased data analytics company Inkiru.
  • We captured the most prolific Tweeters. We could expand on this data to include influencer scores and reach out to the most influential posters.
  • We captured sentiment on all of the tweets. In a friendly forum like HP Discover, we expect the majority of the tweets to be neutral or positive in nature.

HP Vertica Pulse is a result of an ongoing collaboration with HP Labs and is built on Labs’s Live Customer Intelligence (LCI) technology. The Labs team has already had great success with LCI as evidenced in part by their Awards Meter application. HP Vertica has also built a social media connector that loads tweets of interest directly from Twitter into the HP Vertica Analytics Platform, allowing you to start understanding your community right away.

HP Vertica Pulse is yet another example of how you can use the HP Vertica Analytics Platform to bring analytics to the data, speeding analysis time and saving the effort of transferring your data to an external system. Because HP Vertica Pulse is in-database, you can store your text data and the associated sentiment alongside sales, demographic, and other business data. HP Vertica Pulse will help you to harness the voice of your customer so that you can better serve them.

We are now accepting applications to trial Pulse in our private beta. To participate, contact me, Geeta Aggarwal, at gaggarwal@vertica.com.

BDOC – Big Data on Campus

I had a great time speaking at the MIT Sloan Sports Analytics Conference yesterday, and perhaps the most gratifying part of doing a panel in front of a packed house was how many students were in the audience. Having been a bit of a ‘stats geek’ during my college years, I can assure you that such an event, even with a sports theme, would never have drawn such an audience back then.

It was even more gratifying to read this weekend’s Wall Street Journal, with the title Data Crunchers Now The Cool Kids on Campus. Clearly this a terrific time to be studying – and teaching – statistics and Big Data. To quote the article:

The explosive growth in data available to businesses and researchers has brought a surge in demand for people able to interpret and apply the vast new swaths of information, from the analysis of high-resolution medical images to improving the results of Internet search engines.

Schools have rushed to keep pace, offering college-level courses to high-school students, while colleges are teaching intro stats in packed lecture halls and expanding statistics departments when the budget allows.

 

Of course, Big Data training is not just for college students, and at HP Vertica we are working on programs to train both professionals as well as students in conjunction with our colleagues in the HP ExpertOne program. We invite those interested in learning more to contact us – including educational institutions who are interested in adding Big Data training to their curriculum.

Recapping the HP Vertica Boston Meet-Up

This week, some of our Boston-area HP Vertica users joined our team at the HP Vertica office in Cambridge, MA. Over some drinks and great food, we had the honor of hearing from HP Vertica power users Michal Klos followed by Andrew Rollins of Localytics. Both Michal and Andrew offered some valuable insight into how their businesses use the HP Vertica Analytics Platform on Amazon Web Service (AWS).

Michal uses the HP Vertica installation in the cloud, hosted on AWS. The highlight of Michal’s presentation was a live demonstration of a Python script using Fabric (a Python library and command-line tool) and Boto (Python interface to AWS) that executed code to quickly set up and deploy a Vertica cluster in AWS. Launching nodes on the HP Vertica Analytics Platform in AWS eliminates the need to acquire hardware and allows for an extremely speedy deployment. Michal was very complimentary of the recent enhancements to our AWS capabilities in the recently-released version 6.1 of the HP Vertica software.

Michael Klos Demonstration

Following Michal’s demonstration, Andrew took the floor to talk about how Localytics uses the HP Vertica Analytics Platform to  analyze user behavior in mobile and tablet apps.  With HP Vertica, Localytics gives their customers access to granular detail in real-time. Localytics caters to their clients by launching a dedicated node in the cloud for each customer. With the HP Vertica Analytics Platform powering their data in AWS, their customers can start gathering insightful data almost immediately.

Our engineers then took the stage to serve as a panel for questions from the floor. It’s not often that our engineers get the opportunity to answer questions from customers and interested BI professionals in an open forum discussion. Everyone took full advantage of the occasion, asking a number of questions about upcoming features and current use cases.  In addition, our engineers were able to highlight a number of new features from the 6.1 release that the users in attendance may not have been taking advantage of yet.

Meet-ups serve as a fantastic catalyst for users and future users to interact with each other, share best practices and have a valuable conversation with different members of the HP Vertica team. We reiterate our thanks to Michal and Andrew, and to all those that joined us at our offices — thank you for an excellent meet- up!

Don’t miss another valuable opportunity to hear from fellow HP Vertica user Chris Wegrzyn of the Democratic National Committee on our January 24th webinar at 1PM EST. We will discuss how the HP Vertica Analytics Platform revolutionized the way a presidential campaign is run. Register now!

The growth of Big Data, the demand for Data Scientists, and the power of Community

These was an interesting article in CIO last week, IT Departments Battle for Data Analytics Talent, which argues (along with a related McKinsey report) that by 2018 the US will be facing a massive shortage of analytics talent:

By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

On a more personal note, I attended a holiday party this weekend where a parent was relating to me how their pre-college-age son was being advised to pursue ‘Data Scientist’ as a course of study because it is ‘hot’ (while asking me what exactly a ‘Data Scientist’ does).

But is the solution really just to throw more people at the problem? More importantly, is harnessing and leveraging Big Data really a labor problem, a technology problem, or a community problem?

At HP Vertica, we believe that the Big Data challenge will be met – and while we agree that Data Scientist will indeed be one of the hottest (if not sexiest) jobs of the 21st century, we are also confident that the power of community will allow companies to leverage technology to compensate for the demand for labor. Consequently, we have been making significant investments in the MyVertica community and have big plans in store for 2013.

Our friends at the Community Roundtable have put together a terrific set of materials around what it takes to build an active, engaged community – which aligns very well with our efforts to ‘socialize’ our organization.

Watch for much more in the year ahead, and if you’re not already a member of MyVertica, sign up today!

Get Started With Vertica Today

Subscribe to Vertica