verticapy.vDataFrame.score¶

vDataFrame.score(y_true: str, y_score: str, metric: Literal['aic', 'bic', 'accuracy', 'acc', 'balanced_accuracy', 'ba', 'auc', 'roc_auc', 'prc_auc', 'best_cutoff', 'best_threshold', 'false_discovery_rate', 'fdr', 'false_omission_rate', 'for', 'false_negative_rate', 'fnr', 'false_positive_rate', 'fpr', 'recall', 'tpr', 'precision', 'ppv', 'specificity', 'tnr', 'negative_predictive_value', 'npv', 'negative_likelihood_ratio', 'lr-', 'positive_likelihood_ratio', 'lr+', 'diagnostic_odds_ratio', 'dor', 'log_loss', 'logloss', 'f1', 'f1_score', 'mcc', 'bm', 'informedness', 'mk', 'markedness', 'ts', 'csi', 'critical_success_index', 'fowlkes_mallows_index', 'fm', 'prevalence_threshold', 'pm', 'confusion_matrix', 'classification_report', 'r2', 'rsquared', 'mae', 'mean_absolute_error', 'mse', 'mean_squared_error', 'msle', 'mean_squared_log_error', 'max', 'max_error', 'median', 'median_absolute_error', 'var', 'explained_variance']) → float¶

Computes the score using the input columns and the input metric.

Parameters¶

y_true: str

Response column.

y_score: str

Prediction.

metric: str

The metric used to compute the score.

For Classification

accuracy:
Accuracy.

\[Accuracy = \frac{TP + TN}{TP + TN + FP + FN}\]
auc:
Area Under the Curve (ROC).

\[AUC = \int_{0}^{1} TPR(FPR) \, dFPR\]
ba:
Balanced Accuracy.

\[BA = \frac{TPR + TNR}{2}\]
best_cutoff:
Cutoff which optimised the ROC Curve prediction.
bm:
Informedness.

\[BA = TPR + TNR - 1\]
csi:
Critical Success Index.

\[index = \frac{TP}{TP + FN + FP}\]
f1:
F1 Score

\[F_1 Score = 2 \times \frac{Precision \times Recall}{Precision + Recall}\]
fdr:
False Discovery Rate.

\[FDR = 1 - PPV\]
fm:
Fowlkes-Mallows index.

\[FM = \sqrt{PPV * TPR}\]
fnr:
False Negative Rate.

\[FNR = \frac{FN}{FN + TP}\]
for:
False Omission Rate.

\[FOR = 1 - NPV\]
fpr:
False Positive Rate.

\[\frac{FP}{FP + TN}\]
logloss:
Log Loss.

\[Loss = -\frac{1}{N} \sum_{i=1}^{N} \left( y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right)\]
lr+:
Positive Likelihood Ratio.

\[LR+ = \frac{TPR}{FPR}\]
lr-:
Negative Likelihood Ratio.

\[LR- = \frac{FNR}{TNR}\]
dor:
Diagnostic Odds Ratio.

\[DOR = \frac{TP \times TN}{FP \times FN}\]
mcc:
Matthews Correlation Coefficient.

\[MCC = \frac{TP \times TN - FP \times FN}{\sqrt{(TP + FP)(TP + FN)(TN + FP)(TN + FN)}}\]
mk:
Markedness.

\[MK = PPV + NPV - 1\]
npv:
Negative Predictive Value.

\[NPV = \frac{TN}{TN + FN}\]
prc_auc:
Area Under the Curve (PRC).

\[AUC = \int_{0}^{1} Precision(Recall) \, dRecall\]
precision:
Precision.

\[Precision = TP / (TP + FP)\]
pt:
Prevalence Threshold.

\[threshold = \frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]
recall:
Recall.

\[Recall = \frac{TP}{TP + FN}\]
specificity:
Specificity.

\[Specificity = \frac{TN}{TN + FP}\]

For Regression

max:
Max Error.

\[ME = \max_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
mae:
Mean Absolute Error.

\[MAE = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
median:
Median Absolute Error.

\[MedAE = \text{median}_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]
mse:
Mean Squared Error.

\[MSE = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2\]
msle:
Mean Squared Log Error.

\[MSLE = \frac{1}{n} \sum_{i=1}^{n} (\log(1 + y_i) - \log(1 + \hat{y}_i))^2\]
r2:
R squared coefficient.

\[R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}\]
var:
Explained Variance.

\[\text{Explained Variance} = 1 - \frac{Var(y - \hat{y})}{Var(y)}\]

Returns¶

float: score.

Examples¶

Let us build a quick ML model and calculate the score of its predictions.

Load data for machine learning¶

For this example, we will use the winequality dataset.

import verticapy.datasets as vpd

data = vpd.load_winequality()

	123 fixed_acidity Numeric(8)	123 volatile_acidity Numeric(9)	123 citric_acid Numeric(8)	123 residual_sugar Numeric(9)	123 chlorides Float(22)	123 free_sulfur_dioxide Numeric(9)	123 total_sulfur_dioxide Numeric(9)	123 density Float(22)	123 pH Numeric(8)	123 sulphates Numeric(8)	123 alcohol Float(22)	123 quality Integer	123 good Integer	Abc color Varchar(20)
1	3.9	0.225	0.4	4.2	0.03	29.0	118.0	0.989	3.57	0.36	12.8	8	1	white
2	4.7	0.335	0.14	1.3	0.036	69.0	168.0	0.99212	3.47	0.46	10.5	5	0	white
3	4.7	0.455	0.18	1.9	0.036	33.0	106.0	0.98746	3.21	0.83	14.0	7	1	white
4	4.7	0.785	0.0	3.4	0.036	23.0	134.0	0.98981	3.53	0.92	13.8	6	0	white
5	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
6	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
7	4.9	0.42	0.0	2.1	0.048	16.0	42.0	0.99154	3.71	0.74	14.0	7	1	red
8	5.0	0.27	0.4	1.2	0.076	42.0	124.0	0.99204	3.32	0.47	10.1	6	0	white
9	5.0	0.31	0.0	6.4	0.046	43.0	166.0	0.994	3.3	0.63	9.9	6	0	white
10	5.0	0.4	0.5	4.3	0.046	29.0	80.0	0.9902	3.49	0.66	13.6	6	0	red
11	5.0	0.44	0.04	18.6	0.039	38.0	128.0	0.9985	3.37	0.57	10.2	6	0	white
12	5.1	0.11	0.32	1.6	0.028	12.0	90.0	0.99008	3.57	0.52	12.2	6	0	white
13	5.1	0.14	0.25	0.7	0.039	15.0	89.0	0.9919	3.22	0.43	9.2	6	0	white
14	5.1	0.165	0.22	5.7	0.047	42.0	146.0	0.9934	3.18	0.55	9.9	6	0	white
15	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
16	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
17	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
18	5.1	0.39	0.21	1.7	0.027	15.0	72.0	0.9894	3.5	0.45	12.5	6	0	white
19	5.2	0.2	0.27	3.2	0.047	16.0	93.0	0.99235	3.44	0.53	10.1	7	1	white
20	5.2	0.21	0.31	1.7	0.048	17.0	61.0	0.98953	3.24	0.37	12.0	7	1	white
21	5.2	0.22	0.46	6.2	0.066	41.0	187.0	0.99362	3.19	0.42	9.73333333333333	5	0	white
22	5.2	0.31	0.2	2.4	0.027	27.0	117.0	0.98886	3.56	0.45	13.0	7	1	white
23	5.2	0.32	0.25	1.8	0.103	13.0	50.0	0.9957	3.38	0.55	9.2	5	0	red
24	5.2	0.34	0.37	6.2	0.031	42.0	133.0	0.99076	3.25	0.41	12.5	6	0	white
25	5.2	0.36	0.02	1.6	0.031	24.0	104.0	0.9896	3.44	0.35	12.2	6	0	white
26	5.2	0.365	0.08	13.5	0.041	37.0	142.0	0.997	3.46	0.39	9.9	6	0	white
27	5.2	0.48	0.04	1.6	0.054	19.0	106.0	0.9927	3.54	0.62	12.2	7	1	red
28	5.2	0.5	0.18	2.0	0.036	23.0	129.0	0.98949	3.36	0.77	13.4	7	1	white
29	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
30	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
31	5.3	0.165	0.24	1.1	0.051	25.0	105.0	0.9925	3.32	0.47	9.1	5	0	white
32	5.3	0.23	0.56	0.9	0.041	46.0	141.0	0.99119	3.16	0.62	9.7	5	0	white
33	5.3	0.3	0.3	1.2	0.029	25.0	93.0	0.98742	3.31	0.4	13.6	7	1	white
34	5.3	0.33	0.3	1.2	0.048	25.0	119.0	0.99045	3.32	0.62	11.3	6	0	white
35	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
36	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
37	5.3	0.4	0.25	3.9	0.031	45.0	130.0	0.99072	3.31	0.58	11.75	7	1	white
38	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.6	7	1	red
39	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.5666666666667	7	1	red
40	5.3	0.715	0.19	1.5	0.161	7.0	62.0	0.99395	3.62	0.61	11.0	5	0	red
41	5.4	0.22	0.29	1.2	0.045	69.0	152.0	0.99178	3.76	0.63	11.0	7	1	white
42	5.4	0.595	0.1	2.8	0.042	26.0	80.0	0.9932	3.36	0.38	9.3	5	0	white
43	5.4	0.74	0.09	1.7	0.089	16.0	26.0	0.99402	3.67	0.56	11.6	6	0	red
44	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
45	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
46	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
47	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
48	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.65	6	0	white
49	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.6333333333333	6	0	white
50	5.5	0.18	0.22	5.5	0.037	10.0	86.0	0.99156	3.46	0.44	12.2	5	0	white
51	5.5	0.24	0.45	1.7	0.046	22.0	113.0	0.99224	3.22	0.48	10.0	5	0	white
52	5.5	0.29	0.3	1.1	0.022	20.0	110.0	0.98869	3.34	0.38	12.8	7	1	white
53	5.5	0.31	0.29	3.0	0.027	16.0	102.0	0.99067	3.23	0.56	11.2	6	0	white
54	5.5	0.32	0.45	4.9	0.028	25.0	191.0	0.9922	3.51	0.49	11.5	7	1	white
55	5.5	0.35	0.35	1.1	0.045	14.0	167.0	0.992	3.34	0.68	9.9	6	0	white
56	5.5	0.375	0.38	1.7	0.036	17.0	98.0	0.99142	3.29	0.39	10.5	6	0	white
57	5.6	0.15	0.26	5.55	0.051	51.0	139.0	0.99336	3.47	0.5	11.0	6	0	white
58	5.6	0.15	0.31	5.3	0.038	8.0	79.0	0.9923	3.3	0.39	10.5	6	0	white
59	5.6	0.16	0.27	1.4	0.044	53.0	168.0	0.9918	3.28	0.37	10.1	6	0	white
60	5.6	0.175	0.29	0.8	0.043	20.0	67.0	0.99112	3.28	0.48	9.9	6	0	white
61	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
62	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
63	5.6	0.22	0.32	1.2	0.024	29.0	97.0	0.98823	3.2	0.46	13.05	7	1	white
64	5.6	0.26	0.18	1.4	0.034	18.0	135.0	0.99174	3.32	0.35	10.2	6	0	white
65	5.6	0.26	0.26	5.7	0.031	12.0	80.0	0.9923	3.25	0.38	10.8	5	0	white
66	5.6	0.26	0.5	11.4	0.029	25.0	93.0	0.99428	3.23	0.49	10.5	6	0	white
67	5.6	0.28	0.28	4.2	0.044	52.0	158.0	0.992	3.35	0.44	10.7	7	1	white
68	5.6	0.3	0.1	6.4	0.043	34.0	142.0	0.99382	3.14	0.48	9.8	5	0	white
69	5.6	0.35	0.14	5.0	0.046	48.0	198.0	0.9937	3.3	0.71	10.3	5	0	white
70	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
71	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
72	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
73	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
74	5.7	0.15	0.47	11.4	0.035	49.0	128.0	0.99456	3.03	0.34	10.5	8	1	white
75	5.7	0.18	0.26	2.2	0.023	21.0	95.0	0.9893	3.07	0.54	12.3	6	0	white
76	5.7	0.18	0.36	1.2	0.046	9.0	71.0	0.99199	3.7	0.68	10.9	7	1	white
77	5.7	0.2	0.3	2.5	0.046	38.0	125.0	0.99276	3.34	0.5	9.9	6	0	white
78	5.7	0.21	0.32	0.9	0.038	38.0	121.0	0.99074	3.24	0.46	10.6	6	0	white
79	5.7	0.21	0.37	4.5	0.04	58.0	140.0	0.99332	3.29	0.62	10.6	6	0	white
80	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
81	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
82	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
83	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
84	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
85	5.7	0.22	0.29	3.5	0.04	27.0	146.0	0.98999	3.17	0.36	12.1	6	0	white
86	5.7	0.23	0.28	9.65	0.025	26.0	121.0	0.9925	3.28	0.38	11.3	6	0	white
87	5.7	0.25	0.26	12.5	0.049	52.5	106.0	0.99691	3.08	0.45	9.4	6	0	white
88	5.7	0.25	0.26	12.5	0.049	52.5	120.0	0.99691	3.08	0.45	9.4	6	0	white
89	5.7	0.25	0.27	11.5	0.04	24.0	120.0	0.99411	3.33	0.31	10.8	6	0	white
90	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
91	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
92	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
93	5.7	0.27	0.32	1.2	0.046	20.0	155.0	0.9934	3.8	0.41	10.2	6	0	white
94	5.7	0.28	0.24	17.5	0.044	60.0	167.0	0.9989	3.31	0.44	9.4	5	0	white
95	5.7	0.32	0.18	1.4	0.029	26.0	104.0	0.9906	3.44	0.37	11.0	6	0	white
96	5.7	0.32	0.38	4.75	0.033	23.0	94.0	0.991	3.42	0.42	11.8	7	1	white
97	5.7	0.36	0.34	4.2	0.026	21.0	77.0	0.9907	3.41	0.45	11.9	6	0	white
98	5.8	0.14	0.15	6.1	0.042	27.0	123.0	0.99362	3.06	0.6	9.9	6	0	white
99	5.8	0.15	0.32	1.2	0.037	14.0	119.0	0.99137	3.19	0.5	10.2	6	0	white
100	5.8	0.17	0.34	1.8	0.045	96.0	170.0	0.99035	3.38	0.9	11.8	8	1	white

Rows: 1-100 | Columns: 14

Note

VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.

You can easily divide your dataset into training and testing subsets using the vDataFrame.train_test_split() method.

data = vpd.load_winequality()
train, test = data.train_test_split(test_size = 0.2)

Model Initialization¶

First we import the LinearRegression model:

from verticapy.machine_learning.vertica import LinearRegression

Then we can create the model:

model = LinearRegression(
    tol = 1e-6,
    max_iter = 100,
    solver = 'newton',
    fit_intercept = True,
)

Model Training¶

We can now fit the model:

model.fit(
    train,
    [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density"
    ],
    "quality",
    test,
)



=======
details
=======
   predictor    |coefficient|std_err | t_value |p_value 
----------------+-----------+--------+---------+--------
   Intercept    | 147.24228 | 6.79441|21.67109 | 0.00000
 fixed_acidity  |  0.14337  | 0.01214|11.81394 | 0.00000
volatile_acidity| -0.75766  | 0.08602|-8.80825 | 0.00000
  citric_acid   | -0.14234  | 0.09433|-1.50896 | 0.13137
 residual_sugar |  0.04359  | 0.00381|11.44950 | 0.00000
   chlorides    |  0.34561  | 0.38450| 0.89885 | 0.36877
    density     |-143.17375 | 6.91015|-20.71933| 0.00000


==============
regularization
==============
type| lambda 
----+--------
none| 1.00000


===========
call_string
===========
linear_reg('"public"."_verticapy_tmp_linearregression_v_mldb_01f6d4ea97a111efa8720242ac120002_"', '"public"."_verticapy_tmp_view_v_mldb_0250209097a111efa8720242ac120002_"', '"quality"', '"fixed_acidity", "volatile_acidity", "citric_acid", "residual_sugar", "chlorides", "density"'
USING PARAMETERS optimizer='newton', epsilon=1e-06, max_iterations=100, regularization='none', lambda=1, alpha=0.5, fit_intercept=true)

===============
Additional Info
===============
       Name       |Value
------------------+-----
 iteration_count  |  1  
rejected_row_count|  0  
accepted_row_count|5196 

Prediction¶

Prediction is straight-forward:

result = model.predict(
    test,
    [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density"
    ],
    "prediction",
)

	123 fixed_acidity Numeric(8) 100%	...	Abc color Varchar(20) 100%	123 prediction Float(22) 100%
1	4.4	...	white	5.91307882023986
2	4.7	...	white	6.10217388629263
3	5.0	...	white	6.21543697404772
4	5.0	...	white	5.98980806839518
5	5.1	...	white	5.96824546812138
6	5.2	...	white	5.88850865669886
7	5.3	...	white	5.86781988901276
8	5.3	...	white	5.48733242947441
9	5.3	...	white	5.35441920571267
10	5.4	...	white	5.6901866403411
11	5.4	...	white	5.85353634043776
12	5.5	...	white	6.12222700101165
13	5.5	...	white	5.89934987926378
14	5.6	...	white	6.05914608685515
15	5.6	...	white	5.85751774305817
16	5.6	...	white	6.13656376427602
17	5.6	...	white	6.0091700786399
18	5.6	...	white	6.31567725955293
19	5.6	...	white	5.78067125777898
20	5.6	...	white	6.00165359286709

Score¶

Finally we can calculate the scores:

# R2
result.score("quality", "prediction", metric = "r2")
Out[2]: 0.170397882051279

#  MSE
result.score("quality", "prediction", metric = "mse")
Out[3]: 0.672612532820946

#  Max Error
result.score("quality", "prediction", metric = "max")
Out[4]: 3.29668213853051

Note

If the prediction is already part of the dataset, there is no need to use a model to compute a prediction column. Use your column directly.