Commodities¶

This example uses the 'Commodities' dataset to predict the price of different commodities. You can download the Jupyter Notebook of the study here.

• date: Date of the record
• Gold: Price per ounce of Gold
• Oil: Price per Barrel - West Texas Intermediate (WTI)
• Spread: Interest Rate Spreads.
• Vix: The CBOE Volatility Index (VIX) is a measure of expected price fluctuations in the SP500 Index options over the next 30 days.
• Dol_Eur: How much $1 US is in euros.
• SP500: The S&P 500, or simply the S&P, is a stock market index that measures the stock performance of 500 large companies listed on stock exchanges in the United States.

We will follow the data science cycle (Data Exploration - Data Preparation - Data Modeling - Model Evaluation - Model Deployment) to solve this problem.

Initialization¶

This example uses the following version of VerticaPy:

Connect to Vertica. This example uses an existing connection called "VerticaDSN." For details on how to create a connection, use see the connection tutorial.

Let's create a Virtual DataFrame of the dataset.

Data Exploration and Preparation¶

Let's explore the data by displaying descriptive statistics of all the columns.

We have data from January 1986 to the beginning of August 2020. We don't have any missing values, so our data is already clean.

Let's draw the different variables.

Some of the commodities have an upward monotonic trend and some others might be stationary. Let's use Augmented Dickey-Fuller tests to check our hypotheses.

As expected: The price of gold and the S&P 500 index are not stationary. Let's use the Mann-Kendall test to confirm the trends.

Our hypothesis is correct. We can also look at the correlation between the elapsed time and our variables to see the different trends.

In the last plot, it's a bit hard to tell if 'Spread' is stationary. Let's draw it alone.

We can see some sudden changes, so let's smooth the curve.

After each local minimum, there is a local maximum. Let's look at the number of lags needed to keep most of the information. To visualize this, we can draw the autocorrelation function (ACF) and partial autocorrelation function (PACF) plots.

We can clearly see the influence of the last two values on 'Spread,' which makes sense. When the curve slightly changes its direction, it will increase/decrease until reaching a new local maximum/minimum. Only the recent values can help the prediction in case of autoregressive periodical model. The local minimums of interest rate spreads are indicators of an economic crisis.

We saw the correlation between the price-per-barrel of Oil and the time. Let's look at the time series plot of this variable.

Our chart shows effects of the 2010s oil glut and its lowest price in two decades with COVID-19.

Moving on to the correlation matrix, we can see many events that changed drastically the values of commodities, and we know of a correlation between all of them. From here, we could look at how strong this correlation is, which will help us create a model that properly combines all the variable lags in its predictions.

We can see strong correlations between most of the variables. A vector autoregression (VAR) model seems ideal.

Machine Learning¶

Let's create the VAR model to predict the value of various commodities.

Our model is excellent. Let's predict the values these commodities in the near future.

Let's add the prediction to the vDataFrame.

Let's compare the distribution of our prediction and the real value for the variable 'Gold'.

We can see that the distributions are similar.

Conclusion¶

We've solved our problem in a Pandas-like way, all without ever loading data into memory!