Spark Summit East

Posted March 8, 2017 by Soniya Shah, Information Developer

Rear view of two partially unrecognizable men sitting in front of a computer. One of them is pointing at a screen where are several lines of computer code.
This blog post was authored by Myles Collins.

I recently went to the Spark Summit East to take the Spark training and get current on the technology that my group (Vertica Partner Engineering) is using more and more. Conveniently, it was held here in Boston. A few weeks after I registered, marketing decided to sponsor the Summit and asked if, since I was attending anyway, I could present the session that comes with the sponsorship. I find public speaking scary but fun, like a roller coaster ride, so I agreed. The last couple of weeks before the summit I worked with Vertica Product Marketing to get ready.

The first whole day of the three-day conference was Data Science with Spark training. The first lab was a bit tricky because I was not familiar with the Databricks Spark notebook interface. By the second lab I had my feet under me and it went well from then on. Notebooks, once you get used to them, are great for organizing, documenting, and actually executing your code. I’ve always been good about putting comments in my code but notebooks are more like putting your code into your documentation… and executing the code from within your documentation. This was exactly the kind of stuff I wanted to learn.

During a break in class we got to watch the Super Bowl Parade, which was right in front of the building. I took this picture of my man, Tom, as he went by holding one of his five trophies.



The second day was a regular conference day filled with a lot of short 15-30 minute sessions in different tracks. The keynote was about recent trends in Spark and how the industry was moving to meet them. Apparently CPU power/cost is not improving as quickly as IO and storage technology, and this affects the whole industry. I attended a few breakouts based on real-time decision processing, behavior analytics, and genetic data analysis. These weren’t deep dives because the sessions were short, but that kept it lively and interesting.

I presented my session in the afternoon and there were 30-40 people in the room. I’d rehearsed it enough that I could do it in my sleep. I discussed how Vertica works within the open source ecosystem as well as within the database ecosystem as a whole. I especially talked about our new Spark Connector. We scanned the badges of folks that showed up to reach out to them later.

During the rest of the day when I wasn’t attending or giving a session, I’d swing by the Vertica booth and see how my co-workers were doing. If it was busy then I would hang out and help work the booth until it died down. The booth was almost always busy, which is great. I like working the booth at Vertica because people come to ask questions or tell us how much they love Vertica.

Another perk of attending the summit was that there was a special lounge reserved for speakers only and the big shots hung out there. I spoke briefly to Matei Zaharia, who created Spark. Best of all, this room was called the VIP Room, so my ego got a bit bigger every time I went in there.

In the evening, we hosted a small cocktail-and-bowling event at nearby Kings. We chatted with prospects while eating wings and knocking down pins, which was fun.

The next day there was a big snowstorm expected. I got in early before it started and attended several sessions on machine learning, data engineering, and modeling with Spark. During breaks I answered questions and handed out T-shirts at the booth.




By lunch the city was struggling to keep up with the snow (up to 2 inches per hour!), and people were fleeing Boston. There were still some great sessions, so I stayed until the very end. Then, I had to walk a mile to the train in a foot of heavy snow in dress shoes. I had only one all-the-way-to-the-ground awkward wipeout but there were enough witnesses that it un-did the whole VIP Room ego thing.

Thanks to the Spark Summit, there is now a recording available of my 15-minute session on YouTube. Check it out if you want hear how awesomely Vertica fits into the Spark ecosystem.