verticapy.machine_learning.vertica.tsa.MA¶

class verticapy.machine_learning.vertica.tsa.MA(name: str = None, overwrite_model: bool = False, q: int = 1, penalty: Literal[None, 'none', 'l2'] = 'none', C: Annotated[int | float | Decimal, 'Python Numbers'] = 1.0, missing: Literal['drop', 'error', 'zero', 'linear_interpolation'] = 'linear_interpolation')¶

Creates a inDB Moving Average model.

Added in version 11.0.0.

Note

The MA model may be faster and more accurate than ARIMA(0, 0, q) or ARMA(0, q) because the underlying algorithm of MA is quite different.

Parameters¶

name: str, optional

Name of the model. The model is stored in the database.

overwrite_model: bool, optional

If set to True, training a model with the same name as an existing model overwrites the existing model.

q: int, optional

Integer in the range [1, 67), the number of lags to consider in the computation.

penalty: str, optional

Method of regularization.

none:
No regularization.
l2:
L2 regularization.

C: PythonNumber, optional

The regularization parameter value. The value must be zero or non-negative.

missing: str, optional

Method for handling missing values, one of the following strings:

‘drop’:
Missing values are ignored.
‘error’:
Missing values raise an error.
‘zero’:
Missing values are set to zero.
‘linear_interpolation’:
Missing values are replaced by a linearly interpolated value based on the nearest valid entries before and after the missing value. In cases where the first or last values in a dataset are missing, the function errors.

Attributes¶

Many attributes are created during the fitting phase.

theta_: numpy.array: The theta coefficient of the Moving Average process. It signifies the impact and contribution of the lagged error terms in determining the current value within the time series model.
mu_: float: Represents the mean or average of the series. It is a constant term that reflects the expected value of the time series in the absence of any temporal dependencies or influences from past error terms.
mean_: float: The mean of the time series values.
mse_: float: The mean squared error (MSE) of the model, based on one-step forward forecasting, may not always be relevant. Utilizing a full forecasting approach is recommended to compute a more meaningful and comprehensive metric.
n_: int: The number of rows used to fit the model.

Note

All attributes can be accessed using the get_attributes() method.

Note

Several other attributes can be accessed by using the get_vertica_attributes() method.

Examples¶

The following examples provide a basic understanding of usage. For more detailed examples, please refer to the Machine Learning or the Examples section on the website.

Initialization¶

We import verticapy:

import verticapy as vp

Hint

By assigning an alias to verticapy, we mitigate the risk of code collisions with other libraries. This precaution is necessary because verticapy uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from verticapy are used as intended without interfering with functions from other libraries.

For this example, we will generate a dummy time-series dataset that has some noise variation centered around a mean value.

# Initialization
N = 30 # Number of rows
temp = [23] * N
noisy_temp = [x + random.uniform(-5, 5) for x in temp]

# Building the vDataFrame
data = vp.vDataFrame(
    {
        "day": [i for i in range(1, N + 1)],
        "temp": noisy_temp,
    }
)

	123 day Integer	123 temp Numeric(19)
1	1	23.88221621269846
2	2	23.442836386574804
3	3	22.75537104739503
4	4	20.709366402080988
5	5	19.762545529408765
6	6	26.36427149423009
7	7	18.20694475248903
8	8	27.207005567215553
9	9	26.349387301185857
10	10	19.4577993833971
11	11	26.41948064150177
12	12	18.909672485390857
13	13	24.406375208380457
14	14	25.244914292793624
15	15	20.7236659476918
16	16	26.592054514596036
17	17	22.789075431202978
18	18	21.147552722736254
19	19	25.89333208252367
20	20	27.169094658004298
21	21	27.482374731881315
22	22	24.297341430248203
23	23	26.82096843872428
24	24	19.048602116883163
25	25	23.92785146636928
26	26	20.965693684436214
27	27	20.36123527403478
28	28	24.477187986885735
29	29	23.344710250883757
30	30	21.369342467362287

Rows: 1-30 | Columns: 2

Note

VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.

We can plot the data to visually inspect it for the presence of any trends:

data["temp"].plot(ts = "day")

It is obvious there is no trend in our example, but we can confirm it by the mkt() (Mann Kendall test) test:

from verticapy.machine_learning.model_selection.statistical_tests import mkt

mkt(data, column = "temp", ts = "day")

	value
Mann Kendall Test Statistic	0.21409237204201878
S	13.0
STDS	56.0505724026675
p_value	0.830475024124481
Monotonic Trend	❌
Trend	no trend

Rows: 1-6 | Columns: 2

The above report confirms that there is no trend in our data and hence it is stationary. Note the high p-value which is also indicative of the absemce of trend. Once we have established that the data is statioanry, we can then apply MovingAverage model on it.

Model Initialization¶

First we import the MA model:

from verticapy.machine_learning.vertica.tsa import MA

Then we can create the model:

model = MA(q = 2)

Hint

In verticapy 1.0.x and higher, you do not need to specify the model name, as the name is automatically assigned. If you need to re-use the model, you can fetch the model name from the model’s attributes.

Important

The model name is crucial for the model management system and versioning. It’s highly recommended to provide a name if you plan to reuse the model later.

Model Fitting¶

We can now fit the model:

model.fit(data, "day", "temp")


============
coefficients
============
parameter| value  
---------+--------
  alpha  | 0.10401
phi_(t-1)|-0.14893
phi_(t-2)|-0.13359


====
mean
====
predictor| value  
---------+--------
  temp   |23.31761


==================
mean_squared_error
==================
predictor|  value  
---------+---------
  temp   |896.09681


=================
predictor_columns
=================
"temp"

================
timestamp_column
================
day

==============
missing_method
==============
linear_interpolation

===========
call_string
===========
moving_average('"public"."_verticapy_tmp_ma_v_mldb_1d8e7562979e11efa8720242ac120002_"', '"public"."_verticapy_tmp_view_v_mldb_1db72e6c979e11efa8720242ac120002_"', '"temp"', 'day'
USING PARAMETERS q=2, missing=linear_interpolation, regularization='none', lambda=1, compute_mse=true);

===============
Additional Info
===============
       Name       | Value  
------------------+--------
    lag_order     |   2    
  num_predictors  |   1    
      lambda      | 1.00000
rejected_row_count|   0    
accepted_row_count|   30   

Important

To train a model, you can directly use the vDataFrame or the name of the relation stored in the database. The test set is optional and is only used to compute the test metrics. In verticapy, we don’t work using X matrices and y vectors. Instead, we work directly with lists of predictors and the response name.

Model Register¶

In order to register the model for tracking and versioning:

model.register("model_v1")

Please refer to /notebooks/ml/model_tracking_versioning/index.ipynb for more details on model tracking and versioning.

One important thing in time-series forecasting is that it has two types of forecasting:

One-step ahead forecasting
Full forecasting

Important

The default method is one-step ahead forecasting. To use full forecasting, use ``method = “forecast” ``.

One-step ahead¶

In this type of forecasting, the algorithm utilizes the true value of the previous timestamp (t-1) to predict the immediate next timestamp (t). Subsequently, to forecast additional steps into the future (t+1), it relies on the actual value of the immediately preceding timestamp (t).

A notable drawback of this forecasting method is its tendency to exhibit exaggerated accuracy, particularly when predicting more than one step into the future.

Metrics¶

We can get the entire report using:

model.report(start = 3)

	value
explained_variance	-0.0732542924336168
max_error	5.19350701285856
median_absolute_error	2.86551466452421
mean_absolute_error	2.7834259595659
mean_squared_error	9.57513273721072
root_mean_squared_error	3.09437113759981
r2	-0.0749358845074073
r2_adj	-0.117933319887704
aic	65.83090707568
bic	67.5892474743553

Rows: 1-10 | Columns: 2

Important

The value for start has to be greater than the q value selected for the MA model.

You can also choose the number of predictions and where to start the forecast. For example, the following code will allow you to generate a report with 10 predictions, starting the forecasting process at index 25.

model.report(start = 25, npredictions = 10)

	value
explained_variance	-0.232032038410577
max_error	2.79162523159909
median_absolute_error	1.97651777021694
mean_absolute_error	1.78695132390839
mean_squared_error	3.97734730144587
root_mean_squared_error	1.9943287846907
r2	-0.651163548877241
r2_adj	-1.20155139850299
aic	20.9030754495365
bic	10.1219512744047

Rows: 1-10 | Columns: 2

Important

Most metrics are computed using a single SQL query, but some of them might require multiple SQL queries. Selecting only the necessary metrics in the report can help optimize performance. E.g. model.report(metrics = ["mse", "r2"]).

You can utilize the score() function to calculate various regression metrics, with the explained variance being the default.

model.score(start = 25, npredictions = 10)
Out[5]: -0.651163548877241

Important

If you do not specify a starting point and the number of predictions, the forecast will begin at one-fourth of the dataset, which can result in an inaccurate score, especially for large datasets. It’s important to choose these parameters carefully.

Prediction¶

Prediction is straight-forward:

model.predict()

	123 prediction Float(22)
1	23.1350761496582
2	23.1341659920082
3	23.3699131776057
4	23.4049023121203
5	23.441606626231
6	23.4517473051139
7	23.458160890413
8	23.4604707739402
9	23.4616715786085
10	23.4621589942141

Rows: 1-10 | Columns: 2

Hint

You can control the number of prediction steps by changing the npredictions parameter: model.predict(npredictions = 30).

Note

Predictions can be made automatically by using the training set, in which case you don’t need to specify the predictors. Alternatively, you can pass only the vDataFrame to the predict() function, but in this case, it’s essential that the column names of the vDataFrame match the predictors and response name in the model.

If you would like to have the ‘time-stamps’ (ts) in the output then you can switch the output_estimated_ts the parameter.

model.predict(output_estimated_ts = True)

	123 day Float(22)	123 prediction Float(22)
1	31.0	23.1350761496582
2	32.0	23.1341659920082
3	33.0	23.3699131776057
4	34.0	23.4049023121203
5	35.0	23.441606626231
6	36.0	23.4517473051139
7	37.0	23.458160890413
8	38.0	23.4604707739402
9	39.0	23.4616715786085
10	40.0	23.4621589942141

Rows: 1-10 | Columns: 2

Important

The output_estimated_ts parameter provides an estimation of ‘ts’ assuming that ‘ts’ is regularly spaced.

If you don’t provide any input, the function will begin forecasting after the last known value. If you want to forecast starting from a specific value within the input dataset or another dataset, you can use the following syntax.

model.predict(
    data,
    "day",
    "temp",
    start = 25,
    npredictions = 10,
    output_estimated_ts = True,
)

	123 day Float(22)	123 prediction Float(22)
1	26.0	22.9422114546532
2	27.0	23.1528605056339
3	28.0	22.6671242901274
4	29.0	23.1993796805335
5	30.0	23.5805618179796
6	31.0	23.1350761496582
7	32.0	23.1341659920082
8	33.0	23.3699131776057
9	34.0	23.4049023121203
10	35.0	23.441606626231

Rows: 1-10 | Columns: 2

Plots¶

We can conveniently plot the predictions on a line plot to observe the efficacy of our model:

model.plot(data, "day", "temp", npredictions = 15, start=25)

Note

You can control the number of prediction steps by changing the npredictions parameter: model.plot(npredictions = 30).

Please refer to Machine Learning - Time Series Plots for more examples.

Model Register¶

In order to register the model for tracking and versioning:

model.register("model_v1")

Please refer to /notebooks/ml/model_tracking_versioning/index.ipynb for more details on model tracking and versioning.

Full forecasting¶

In this forecasting approach, the algorithm relies solely on a chosen true value for initiation. Subsequently, all predictions are established based on a series of previously predicted values.

This methodology aligns the accuracy of predictions more closely with reality. In practical forecasting scenarios, the goal is to predict all future steps, and this technique ensures a progressive sequence of predictions.

Metrics¶

We can get the report using:

model.report(start = 25, method = "forecast")

By selecting start = 25, we will measure the accuracy from 40th time-stamp and continue the assessment until the last available time-stamp.

	value
explained_variance	0.0569168245777847
max_error	3.08599551248605
median_absolute_error	1.97651777021694
mean_absolute_error	1.66715805661204
mean_squared_error	3.79099742881507
root_mean_squared_error	1.94704838892491
r2	-0.573801907133249
r2_adj	-1.09840254284433
aic	20.6631457915001
bic	9.88202161636826

Rows: 1-10 | Columns: 2

Notice that the accuracy using method = forecast is poorer than the one-step ahead forecasting.

You can utilize the score() function to calculate various regression metrics, with the explained variance being the default.

model.score(start = 25, npredictions = 30, method = "forecast")
Out[6]: -0.573801907133249

Prediction¶

Prediction is straight-forward:

model.predict(start = 25, npredictions = 15, method = "forecast")

	123 prediction Float(22)
1	22.9422114546532
2	23.4472307865208
3	23.3907744864646
4	23.449831303896
5	23.4510849142861
6	23.4591609651628
7	23.4605312306806
8	23.4618141817261
9	23.4621883089971
10	23.4624154174395
11	23.4624992208785
12	23.4625420412502
13	23.4625596138747
14	23.4625679513663
15	23.4625715406113

Rows: 1-15 | Columns: 2

If you want to forecast starting from a specific value within the input dataset or another dataset, you can use the following syntax.

model.predict(
    data,
    "day",
    "temp",
    start = 25,
    npredictions = 20,
    output_estimated_ts = True,
    output_standard_errors = True,
    method = "forecast",
)

	123 day Float(22)	123 prediction Float(22)
1	26.0	22.9422114546532
2	27.0	23.4472307865208
3	28.0	23.3907744864646
4	29.0	23.449831303896
5	30.0	23.4510849142861
6	31.0	23.4591609651628
7	32.0	23.4605312306806
8	33.0	23.4618141817261
9	34.0	23.4621883089971
10	35.0	23.4624154174395
11	36.0	23.4624992208785
12	37.0	23.4625420412502
13	38.0	23.4625596138747
14	39.0	23.4625679513663
15	40.0	23.4625715406113
16	41.0	23.462573188969
17	42.0	23.4625739139491
18	43.0	23.4625742421257
19	44.0	23.4625743878517
20	45.0	23.4625744533961

Rows: 1-20 | Columns: 2

Plots¶

We can conveniently plot the predictions on a line plot to observe the efficacy of our model:

model.plot(
    data,
    "day",
    "temp",
    npredictions = 15,
    start = 25,
    method = "forecast",
)

__init__(name: str = None, overwrite_model: bool = False, q: int = 1, penalty: Literal[None, 'none', 'l2'] = 'none', C: Annotated[int | float | Decimal, 'Python Numbers'] = 1.0, missing: Literal['drop', 'error', 'zero', 'linear_interpolation'] = 'linear_interpolation') → None¶: Must be overridden in the child class

Methods

`__init__`([name, overwrite_model, q, ...])	Must be overridden in the child class
`contour`([nbins, chart])	Draws the model's contour plot.
`deploySQL`([ts, y, start, npredictions, ...])	Returns the SQL code needed to deploy the model.
`does_model_exists`(name[, raise_error, ...])	Checks whether the model is stored in the Vertica database.
`drop`()	Drops the model from the Vertica database.
`export_models`(name, path[, kind])	Exports machine learning models.
`features_importance`([idx, show, chart])	Computes the model's features importance.
`fit`(input_relation, ts, y[, test_relation, ...])	Trains the model.
`get_attributes`([attr_name])	Returns the model attributes.
`get_match_index`(x, col_list[, str_check])	Returns the matching index.
`get_params`()	Returns the parameters of the model.
`get_plotting_lib`([class_name, chart, ...])	Returns the first available library (Plotly, Matplotlib, or Highcharts) to draw a specific graphic.
`get_vertica_attributes`([attr_name])	Returns the model Vertica attributes.
`import_models`(path[, schema, kind])	Imports machine learning models.
`plot`([vdf, ts, y, start, npredictions, ...])	Draws the model.
`predict`([vdf, ts, y, start, npredictions, ...])	Predicts using the input relation.
`register`(registered_name[, raise_error])	Registers the model and adds it to in-DB Model versioning environment with a status of 'under_review'.
`regression_report`([metrics, start, ...])	Computes a regression report using multiple metrics to evaluate the model (`r2`, `mse`, `max error`...).
`report`([metrics, start, npredictions, method])	Computes a regression report using multiple metrics to evaluate the model (`r2`, `mse`, `max error`...).
`score`([metric, start, npredictions, method])	Computes the model score.
`set_params`([parameters])	Sets the parameters of the model.
`summarize`()	Summarizes the model.
`to_binary`(path)	Exports the model to the Vertica Binary format.
`to_pmml`(path)	Exports the model to PMML.
`to_python`([return_proba, ...])	Returns the Python function needed for in-memory scoring without using built-in Vertica functions.
`to_sql`([X, return_proba, ...])	Returns the SQL code needed to deploy the model without using built-in Vertica functions.
`to_tf`(path)	Exports the model to the Frozen Graph format (TensorFlow).

Attributes

object_type