Vertica Blog

Vertica Blog

Streaming

DX Radio Episode 3: Climate Corporation and Digital Agriculture

I have the honor of hosting the latest episode of DX Radio. It features The Climate Corporation’s Daniel McCaffrey, Vice president of Data and Analytics, and Erich Hochmuth, Big Data Architect Engineer, as they explore the digital transformation of farming.

Constellation Research Report Finds Vertica 10 a Leading Choice

A few pages into a new, detailed overview of Vertica 10, produced by Constellation Research, principle analyst Doug Henschen summarizes the top new capabilities of Vertica 10 like this: In short, the Vertica team has steadily evolved the product to keep pace with the demands of organizations that seek to do the following: Unify diverse...

Take Nothing but Memories. Leave Nothing but Footprints

For those who have read my previous blog postings, attended one of our Big Data & Machine Learning Meetups, or have met me at one of the many trade shows or conferences over the years, you will be all too aware of my love for tracking aircraft using a Raspberry Pi, Kafka and Vertica. For...
The Earth

“awk” sed Grep, “whoami?”. Time CC if bash makes head or tail and sort > plain.txt and make us API

Things to do whilst confined to barracks due to COVID-19 For those who know me well, having worked in IT for four decades, will know that I can be a bit of a Luddite when it comes to new-fangled technology. Give me a character-based, green-screen PuTTY application over any pointy-clicky interface.  Let me use vsql...
Flight Tracking

Pennard, We Have a Problem – Troubleshooting Flight Tracking with the Grafana Plugin for Vertica

With Vertica’s Management Console, I have everything I need at my fingertips to manage and monitor my Vertica clusters and databases.  Whether that be investigating query performance issues, checking on the health of the cluster’s nodes, running the Database Designer or marveling at the volume of data being ingested by the Vertica Scheduler from a...
Planets in solar system

The Kafka Convergence

Apache Kafka seems to be everywhere these days. I’m feeling a bit surrounded by Kafka right now, in fact, like I’m at the center of a convergence of the planets, and data flowing through Kafka is the gravitational pull. Let me give you an idea of what I mean. So, Vertica just announced the release...
Santa flying over night sky

Tracking Santa’s Flight Plan

You probably all know of my love for Live Flight tracking using a Raspberry Pi (or more precisely a plethora of them) and Vertica. Although I started flight tracking around 2012, my interest grew when I saw this from NATS (National Air Traffic Services) https://vimeo.com/110348926 - a series of great video clips from UK, Middle...

Vertica on the Road! Using Vertica to Prepare and Analyze Vehicle Telematics Data

Vehicles generate a lot of data from onboard electronics and there is increasing interest in monitoring vehicle handling to help assess insurance risk and premiums. At Vertica, we're working with a few auto manufacturers, so I built this demo to show a relatively simple example of data collection, aggregation and preparation, and analysis of the...
Machine learning robot studying equations and graphs

1 – Why Would You Do Machine Learning in a Database?

Co-authored with Paige Roberts. A lot of modern scalable analytical databases like Vertica allow you to do machine learning data analytics from end to end, right in the database, rather than moving and transforming the data first into something like a Spark dataframe or a Python data structure. Whenever people hear about this capability, they...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 7

Preparing Vertica to consume messages from Kafka How time has flown (no pun intended!).  At long last getting back to continuing this project on tracking commercial aircraft… Having covered ADS-B, the Raspberry Pi, DUMP1090, Zookeeper and Kafka and then in the previous blog post, how to develop an Extract, Transform and Load (ETL) application to...
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time – Part 6

Part 6: Extract, Transform and Load ADS-B messages into Kafka I have discussed in previous blog posts the continuous stream of messages from aircraft transponders, captured and decoded using the DUMP1090 application, which we are planning on feeding into a series of Kafka topics, prior to loading into their corresponding tables in a Vertica database....
Commercial passenger plane with Vertica painted on the tail

Blog Post Series: Using Vertica to Track Commercial Aircraft in near Real-Time — Part 5

Part Five: Apache Kafka As discussed in an earlier blog post, we have a continuous stream of messages from aircraft transponders captured and decoded using the DUMP1090 application. We now want to load this data into a series of Vertica tables. As with the streaming nature of the incoming data, with many 10s, possibly 100s...