Vertica Blog

Vertica Blog

Streaming

Machine learning robot studying equations and graphs

In-Database Machine Learning 1 – Why Would You Do That?

Co-authored with Paige Roberts. A lot of modern like Vertica allow you to do machine learning from end to end, right in the database, rather than moving and transforming the data first into something like a Spark dataframe or a Python data structure. Whenever people hear about this capability, they have two questions. The first...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 7

Preparing Vertica to consume messages from Kafka How time has flown (no pun intended!).  At long last getting back to continuing this project on tracking commercial aircraft… Having covered ADS-B, the Raspberry Pi, DUMP1090, Zookeeper and Kafka and then in the how to develop an Extract, Transform and Load (ETL) application to present a continuous...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time – Part 6

Part 6: Extract, Transform and Load ADS-B messages into Kafka I have discussed in previous blog posts the continuous stream of messages from aircraft transponders, captured and decoded using the DUMP1090 application, which we are planning on feeding into a series of Kafka topics, prior to loading into their corresponding tables in a Vertica database....
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 5

Part Five: Apache Kafka As discussed in an , we have a continuous stream of messages from aircraft transponders captured and decoded using the DUMP1090 application. We now want to load this data into a series of Vertica tables. As with the streaming nature of the incoming data, with many 10s, possibly 100s of messages...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 4

Part Four: Installing DUMP1090 to decode ADS-B messages The detailed the hardware requirements used in this project for tracking commercial aircraft in near real-time. In this blog post I will touch on installing the operating system on the Raspberry Pi (RPI) and the DUMP1090 software used for decoding the ADS-B signals being received by the...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 3

Building a Raspberry Pi to Capture ADS-B signals Picking apples, pears, blackberries or raspberries In a , I provided a very high-level overview of ADS-B, and that with the appropriate pieces of hardware and some open-source software, it was possible to capture and decode the radio signals being broadcast from commercial aircraft, with the view...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real Time –Part 2

Part Two: Automatic Dependent Surveillance - Broadcast (ADS-B) A Source of Streaming Data If you're joining us a new reader, be sure to read of this series to get up to speed! When looking for a topic to use in the first of The Lab Series’ mini projects for the Big Data and Machine Learning...

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real Time — Part 1

Project Overview This post presents an overview of an ongoing series that focuses on using Vertica, Raspberry Pi, and Apache Kafka to track commercial aircraft in near-real time. Often, users try to comprehend the many advanced capabilities of Vertica. Doing so can present some difficulty, especially if you are alone. Whether you are identifying useful...