Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real Time
A Source of Streaming DataIf you’re joining us a new reader, be sure to read part one of this series to get up to speed!
When looking for a topic to use in the first of The Lab Series’ mini projects for the Big Data and Machine Learning Meetups in London (and latterly in Cambridge and Munich), I decided to continue with a personal project that I started a number of years ago.
This project revolved around tracking commercial aircraft in near real-time, using a Raspberry Pi computer, a digital broadcast receiver and some open source software called DUMP1090.
One of the requirements for the Meetups’ mini project was to find a source of streaming data that would be readily available from anywhere in the world, and could be used within the project as it developed over a number of Meetup events. With the name of the Meetup being Big Data and Machine Learning, we naturally wanted a source of data that could demonstrate the many capabilities of Vertica – especially as we moved towards advanced analytics and machine learning.
Another requirement of finding a source of streaming data was that it was “free”!
RADAR and ADS-BThere are many ways to track aircraft as they fly in the skies above us. From the early 1900s, radar equipment has been used to determine the range, angle and velocity of objects – aircraft, ships, guided missiles, motor vehicles, weather formations and the terrain.
As the number of aircraft increased, and technology has advanced, more sophisticated methods were developed for tracking aircraft.
The technology we will be concentrating on for this project is named Automatic Dependent Surveillance – Broadcast (ADS-B).
ADS-B is a surveillance technology in which an aircraft determines its position via satellite navigation systems and periodically broadcasts it, along with other data points from a transponder within the aircraft. This is broadcast (or transmitted) over the radio waves on a frequency of 1090 MHz.
ADS-B is “Automatic” in that it requires no manual intervention. It is “Dependent” in that is relies upon the aircraft’s navigation systems and other on-board equipment to provide the data. By incorporating the aircraft’s position via satellite navigation systems, and with other data is “Broadcast” via the radio waves, it is used for “Surveillance” by both ground-based stations and other aircraft for tracking, collision avoidance and many other applications.
RADAR and ADS-B are far more complex a subject matter than can be given justice to here, but suffice to say, with the appropriate hardware to receive the 1090 MHz radio signals and software to decode the Software Defined Radio (SDR) messages, it is possible for anyone to establish a platform for tracking aircraft.
For those who want to know more about Radio Detection And Ranging (aka RADAR) and ADS-B, there many resources available (e.g. https://en.wikipedia.org/wiki/Radar and https://en.wikipedia.org/wiki/Automatic_dependent_surveillance_%E2%80%93_broadcast).
With a smidgen of information on what ADS-B is, the next blog post in this series goes on to describe how to set up a Raspberry Pi to capture signals from aircraft transponders.