Vertica

Archive for the ‘interns’ Category

Data Quest – Sifting through terabytes of diagnostic system data

All of the interns at Vertica have found interesting and useful projects to work on during their stay. The focus of my efforts has been the Data Quest project.

At Vertica, an important goal is to keep customers happy. In order to do this effectively it’s necessary to know what it is a customer wants even if they aren’t sure themselves. A potential source of this information comes in the form of diagnostic data which can tell us how Vertica is actually used in the wild. Internally, the Vertica Analytics Platform tracks a wealth of information about its state and usage in over 300 system tables. If something goes wrong, customers have the option of creating a diagnostics data dump file of the current states of these system tables so that this information can be used to resolve problems and improve future experiences. As of this writing there is over 515 dump files available having over 100Gb of compressed data.

The Data Quest goal has been to take all this Vertica meta-data, pump it into a database and get cracking on the analysis. Unfortunately, getting this data ready for analysis is a nontrivial task. A number of hurdles had to be crossed to get the data ready for loading an analysis.

(more…)

Fun with Vertica UDX

During a week in July, Vertica held a competition for all of us, the 2012 summer interns. We had a week to write a user defined function for Vertica. We could work individually or in teams of two, and the function could do whatever we wanted it to do. We were, however, told that we would be judged by a range of people from across the company on criteria that was never quite explained to us.

Like most people, and probably many Vertica customers, I had never written a UDX for Vertica before. I was already in the middle of my normal summer work, and did not necessarily relish the idea of putting off my other tasks, which I was enjoying, to learn a new skill that seemed very specific. I was still not sure what exactly a UDX was, but writing one sounded difficult and unpleasant.

So, being the natural procrastinator that I am, I put the competition off. Unfortunately for me, this summer Vertica has an odd number of engineering interns and my fellow interns did not seem as daunted as I was. In the time that I spent pretending that the competition was not actually occurring, they had already divided into teams and started working on their ideas. The final presentation was a mere couple days away, and I needed an idea quickly. Because I also was now apparently working alone, I needed that idea to be easy to implement.

(more…)

Heating up Data in Vertica

A couple of weeks ago Vertica saw the interns teaming up and working on their own UDx projects. These projects were then presented under the eyes of judges who scored each group on various criteria. One hot project was our Heat Map UDT. After some additional work, the Heat Map is becoming a promising addition to the Vertica UDx family!  Let’s get this show on the road!

Columns of data in need of some quick and dirty analysis?  Enter Vertica and the Heat Map Transform, a highly extensible and parameterized analysis tool.

Imagine you had a client who came to you for advice on how to improve their popular first person shooter.  After some talking you come to the conclusion that the maps may not be well balanced and players simply die too often.  If only there was some way to keep them alive longer, and still have an exciting and fast-paced level!  A good first step might be to figure out where the dangerous zones are in the maps so that the client may figure out where the maps may need balance changes.  Perhaps by balancing out the map, players will start to utilize the level in its entirety and allow for more tactical, strategic play.  Conveniently, the client has been logging all of the death and killing locations into his Vertica database. Now how to quickly process this information to get some useful visual results?  Easy! Let’s try out the Heat Map UDT from the Vertica Extension Packages GitHub repository.

(more…)

On Both Sides of the Internship

Author: Lyric Doshi

Three cheers for Vertica Summer Interns 2011!
.

As summer comes to an end, we bid goodbye to yet another amazing crop of summer interns. This years interns Hieu, Zhongliang, Ruchika and Zhijie were MS/PhD students from different schools along the east coast.  They worked on projects to extend the Vertica SDK, enhance Hadoop/Pig connectivity, and create internal developer productivity tools.  We plan to incorporate much of this work into a future major release of Vertica. In fact, there was so much excitement surrounding their work that in addition to the traditional presentation to engineering, they were asked to present to the entire company after having lunch with Vertica VP&GM Chris Lynch.

Once a picky an intern at other companies (and a very happy one at Vertica, why else would I have come back?), I had the opportunity this year to run our internship program, beginning with coordinating interviews for nearly 40 candidates down to a 4 solid interns. That was over 3 months ago. In the past few days, I took some time to speak with each intern to hear about their experience and I was happy to hear some of my own words to my mentors at Vertica from 2 years ago being reiterated.

Zhongliang gained experience working on a full Java project for first time and told me he felt his coding improved dramatically thanks to feedback from his mentor Matt.

Ruchika told me how her project made some of her friends jealous because she was never bored at work. She appreciated how everyone here dropped their work to help her out when she had questions. She singled out the unwavering patience of her mentor Ben in answering her questions. Having appreciated the same time and time again, I responded, “Been there, done that.”

Hieu highlighted how much fun he had in addition to his project, partaking in our Ping Pong tournament, joining the weekly pick-up basketball, learning the art of sword-play, destroying us in Starcraft, and attending cook-outs hosted by co-workers. He even laughed himself through an unforgettable first experience with water sports (hint: it involved an inner tube, some soccer shorts, and a motorboat) at our annual interns party.


Zhijie was very happy to work on a project that, while sufficiently separated that he did not worry about causing trouble, was in the release plan. Whenever he got frustrated with coding issues, he looked at customer feature requests page to see all the demands for what he was working on and found real inspiration that he was making a difference.

Repeatedly pressing each for complaints in the spirit of constructive criticism, major or minor, I finally forced something out of the Zhijie “The office was a little hot when the AC broke.” Surely a sign of a successful summer?

But it’s not just the interns that got something out of the internship program. Our program is run entirely by engineering and both I and the four dedicated intern mentors gained experience managing projects, goals and expectations. We’ve adapted our program over the years, trying both team and individual projects focussed on everything from tools and demos to server-side changes. This year, we even sat our interns in desks near their respective mentors this year for a proper full-timer experience and even higher mentor accessibility.

A big thank you to all of our interns for your hard work and commitment. We’ll miss having you around but wish you the best for the coming school year and hope to see you again soon!

Get Started With Vertica Today

Subscribe to Vertica