My Intern Experience @ Vertica

Posted September 26, 2018 by Phil Molea, Sr. Information Developer, Vertica

Zherui Cao authored this post. Hello, I am Zherui Cao. I am a currently studying for my Masters Degree in computing and information technology at the University of Pittsburgh. A kind Pitt alumni informed me of a summer internship opportunity at Vertica. With database development experience, I quickly became a good candidate for an internship. And since it was my first exposure to columnar storage database, a very interesting and cool concept, I chose to intern at Vertica for the summer of 2018.

Opportunity

At the start of my internship, my manager spent a lot of time discussing potential projects I could work on over the summer. The projects are carefully chosen and can be completed in about three months. I was very excited to work on real-world projects, rather than writing software for a school assignment. After careful consideration, I chose to work on a project to load/unload Apache orc data into and out of the Vertica database. There are several steps involved in an internship project, including creating a requirements specification and discussing the details with other engineers. The process also includes implementing the project and discussing the results and performance, as well as working with quality assurance to ensure the final project works on an extremely large dataset. Interns are expected to present their ideas and results in a series of presentations. During these presentations, I was able to discuss my ideas with engineers from many different groups. Even when we were not directly discussing the code, my project was helped tremendously because other engineers may see things in a different light, or point out something I may have missed. All discussions resulted in good advice for my project. Since Vertica is an engineer-driven company, where engineers design the details of the requirements, I knew I would be able to put my ideas to work. Therefore, I do know this is not a change to code in a real-world project, but a change to bring my ideas into the real world. During the internship, there is also a Hackathon week where all employees (even non-engineers) spend time to develop ideas to make Vertica a better and more innovative product. The Hackathon allows Vertica employees to present ideas that may be a little off the wall; but this creates discussion and most times we end up with a great idea.

Growth and Mentoring

Working at Vertica provided me an opportunity to show off and reinforce my technical skills. This project was complicated, with large amount of layers of concepts, where I can put my object-oriented programming ability to use. Apart from that, I also did time and space analysis. Since I am very familiar with algorithm and data structure, I could easily handle these and provided good feedback on increasing efficiency on running time and memory space usage. I also learned a great deal from senior engineers, who were very generous in giving advice on both coding and engineering concepts. Every intern is mentored by a senior engineer. My mentor, Deepak Majeti, was very patient and helped me understand the Vertica structure. We all know beginnings can be hard, and learning about Vertica software can be even harder due to the large number of files. I was very lucky to have a mentor such as Deepak who was able to provide excellent direction to get me through this early phase. Since my main project was logically simple, I was able to finish my coding in a short time. However, Vertica is software is famous for its efficiency and fast performance. So, with my main project coding done, I was able to spend a lot of time focusing on reducing runtime. This type of work is challenging and attractive to me. Working on this project, I not only implemented a product feature but, using my computer architecture knowledge, was able to increase software efficiency and performance. Seeing the runtimes get lower and lower was extremely fulfilling.

Culture

Vertica’s Pittsburgh office is a very small team compared to their Boston location. This small office space allowed me to get to know my colleagues very well, and all of them were very happy to help me in many ways. Even though we had a small team, we had team members from Russia, India, China, and Sri Lanka. At time we were able to go out socially after work. For example, we went bowling and axe throwing (with no injuries!). These outings were just as important to me as the time in the office and left me with unforgettable memories.

Conclusion

During the internship, I learned a lot about the columnar data storage format, including Vertica, open source library Apache Parquet, and Apache ORC. Since Vertica’s EON mode has been successful in taking advantage of Cloud computing, I also learned many concepts and technologies about cloud databases and shared file systems. With the advice and help from my teammates, I found it an enjoyable experience to work in a group. Team members present different ideas, some of which efficiently solve problems. Overall, this internship has been very beneficial for my career.