Feedback

×

• Feedback
• Was this page helpful?*
  • Yes
  • No
• Do you have any additional feedback?
  • Yes
  • No
• Additional Feedback

verticapy.machine_learning.model_selection.hp_tuning.grid_search_cv

verticapy.machine_learning.model_selection.hp_tuning.grid_search_cv(estimator: VerticaModel, param_grid: dict | list, input_relation: Annotated[str | vDataFrame, ''], X: Annotated[str | list[str], 'STRING representing one column or a list of columns'], y: str, metric: str = 'auto', cv: int = 3, average: Literal['binary', 'micro', 'macro', 'weighted'] = 'weighted', pos_label: Annotated[bool | float | str | timedelta | datetime, 'Python Scalar'] | None = None, cutoff: Annotated[int | float | Decimal, 'Python Numbers'] = -1, training_score: bool = True, skip_error: bool | Literal['no_print'] = True, print_info: bool = True, **kwargs) → TableSample

Computes the k-fold grid search of an estimator.

estimator: VerticaModel

Vertica estimator with a fit method.

param_grid: dict | list

Dictionary of the parameters to test. It can also be a list of the different combinations.

input_relation: SQLRelation

Relation used to train the model.

X: SQLColumns

list of the predictor columns.

y: str

Response Column.

metric: str, optional

Metric used for the model evaluation.

• auto:

  logloss for classification & RMSE for regression.

For Classification

• accuracy:

  Accuracy.

  \[Accuracy = \frac{TP + TN}{TP + TN + FP + FN}\]

• auc:

  Area Under the Curve (ROC).

  \[AUC = \int_{0}^{1} TPR(FPR) \, dFPR\]

• ba:

  Balanced Accuracy.

  \[BA = \frac{TPR + TNR}{2}\]

• bm:

  Informedness

  \[BM = TPR + TNR - 1\]

• csi:

  Critical Success Index

  \[index = \frac{TP}{TP + FN + FP}\]

• f1:

  F1 Score .. math:

  F_1 Score = 2 \times 
  

rac{Precision times Recall}{Precision + Recall}

• fdr:

  False Discovery Rate

  \[FDR = 1 - PPV\]

• fm:

  Fowlkes-Mallows index

  \[FM = \sqrt{PPV * TPR}\]

• fnr:

  False Negative Rate

  \[FNR = \frac{FN}{FN + TP}\]

• for:

  False Omission Rate

  \[FOR = 1 - NPV\]

• fpr:

  False Positive Rate

  \[FPR = \frac{FP}{FP + TN}\]

• logloss:

  Log Loss

  \[Loss = -\frac{1}{N} \sum_{i=1}^{N} \left( y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right)\]

• lr+:

  Positive Likelihood Ratio.

  \[LR+ = \frac{TPR}{FPR}\]

• lr-:

  Negative Likelihood Ratio.

  \[LR- = \frac{FNR}{TNR}\]

• dor:

  Diagnostic Odds Ratio.

  \[DOR = \frac{TP \times TN}{FP \times FN}\]

• mcc:

  Matthews Correlation Coefficient

• mk:

  Markedness

  \[MK = PPV + NPV - 1\]

• npv:

  Negative Predictive Value

  \[NPV = \frac{TN}{TN + FN}\]

• prc_auc:

  Area Under the Curve (PRC)

  \[AUC = \int_{0}^{1} Precision(Recall) \, dRecall\]

• precision:

  Precision

  \[TP / (TP + FP)\]

• pt:

  Prevalence Threshold.

  \[\frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]

• recall:

  Recall.

  \[TP / (TP + FN)\]

• specificity:

  Specificity.

  \[TN / (TN + FP)\]

For Regression

• max:

  Max Error.

  \[ME = \max_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• mae:

  Mean Absolute Error.

  \[MAE = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• median:

  Median Absolute Error.

  \[MedAE = \text{median}_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• mse:

  Mean Squared Error.

  \[MSE = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2\]

• msle:

  Mean Squared Log Error.

  \[MSLE = \frac{1}{n} \sum_{i=1}^{n} (\log(1 + y_i) - \log(1 + \hat{y}_i))^2\]

• r2:

  R squared coefficient.

  \[R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}\]

• r2a:

  R2 adjusted

  \[\text{Adjusted } R^2 = 1 - \frac{(1 - R^2)(n - 1)}{n - k - 1}\]

• var:

  Explained Variance.

  \[VAR = 1 - \frac{Var(y - \hat{y})}{Var(y)}\]

• rmse:

  Root-mean-squared error

  \[RMSE = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}\]

cv: int, optional

Number of folds.

average: str, optional

The method used to compute the final score for multiclass-classification.

• binary:

  considers one of the classes as positive and use the binary confusion matrix to compute the score.

• micro:

  positive and negative values globally.

• macro:

  average of the score of each class.

• weighted:

  weighted average of the score of each class.

pos_label: PythonScalar, optional

The main class to be considered as positive (classification only).

cutoff: float, optional

The model cutoff (classification only).

training_score: bool, optional

If set to True, the training score is computed with the validation score.

skip_error: bool, optional

If set to True and an error occurs, the error is displayed but not raised.

print_info: bool, optional

If set to True, prints the model information at each step.

TableSample

Result of the grid search.

We import verticapy:

import verticapy as vp

Hint

By assigning an alias to verticapy, we mitigate the risk of code collisions with other libraries. This precaution is necessary because verticapy uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from verticapy are used as intended without interfering with functions from other libraries.

For this example, we will use the Wine Quality dataset.

import verticapy.datasets as vpd

data = vpd.load_winequality()

	123 fixed_acidity Numeric(8)	123 volatile_acidity Numeric(9)	123 citric_acid Numeric(8)	123 residual_sugar Numeric(9)	123 chlorides Float(22)	123 free_sulfur_dioxide Numeric(9)	123 total_sulfur_dioxide Numeric(9)	123 density Float(22)	123 pH Numeric(8)	123 sulphates Numeric(8)	123 alcohol Float(22)	123 quality Integer	123 good Integer	Abc color Varchar(20)
1	3.9	0.225	0.4	4.2	0.03	29.0	118.0	0.989	3.57	0.36	12.8	8	1	white
2	4.7	0.335	0.14	1.3	0.036	69.0	168.0	0.99212	3.47	0.46	10.5	5	0	white
3	4.7	0.455	0.18	1.9	0.036	33.0	106.0	0.98746	3.21	0.83	14.0	7	1	white
4	4.7	0.785	0.0	3.4	0.036	23.0	134.0	0.98981	3.53	0.92	13.8	6	0	white
5	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
6	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
7	4.9	0.42	0.0	2.1	0.048	16.0	42.0	0.99154	3.71	0.74	14.0	7	1	red
8	5.0	0.27	0.4	1.2	0.076	42.0	124.0	0.99204	3.32	0.47	10.1	6	0	white
9	5.0	0.31	0.0	6.4	0.046	43.0	166.0	0.994	3.3	0.63	9.9	6	0	white
10	5.0	0.4	0.5	4.3	0.046	29.0	80.0	0.9902	3.49	0.66	13.6	6	0	red
11	5.0	0.44	0.04	18.6	0.039	38.0	128.0	0.9985	3.37	0.57	10.2	6	0	white
12	5.1	0.11	0.32	1.6	0.028	12.0	90.0	0.99008	3.57	0.52	12.2	6	0	white
13	5.1	0.14	0.25	0.7	0.039	15.0	89.0	0.9919	3.22	0.43	9.2	6	0	white
14	5.1	0.165	0.22	5.7	0.047	42.0	146.0	0.9934	3.18	0.55	9.9	6	0	white
15	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
16	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
17	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
18	5.1	0.39	0.21	1.7	0.027	15.0	72.0	0.9894	3.5	0.45	12.5	6	0	white
19	5.2	0.2	0.27	3.2	0.047	16.0	93.0	0.99235	3.44	0.53	10.1	7	1	white
20	5.2	0.21	0.31	1.7	0.048	17.0	61.0	0.98953	3.24	0.37	12.0	7	1	white
21	5.2	0.22	0.46	6.2	0.066	41.0	187.0	0.99362	3.19	0.42	9.73333333333333	5	0	white
22	5.2	0.31	0.2	2.4	0.027	27.0	117.0	0.98886	3.56	0.45	13.0	7	1	white
23	5.2	0.32	0.25	1.8	0.103	13.0	50.0	0.9957	3.38	0.55	9.2	5	0	red
24	5.2	0.34	0.37	6.2	0.031	42.0	133.0	0.99076	3.25	0.41	12.5	6	0	white
25	5.2	0.36	0.02	1.6	0.031	24.0	104.0	0.9896	3.44	0.35	12.2	6	0	white
26	5.2	0.365	0.08	13.5	0.041	37.0	142.0	0.997	3.46	0.39	9.9	6	0	white
27	5.2	0.48	0.04	1.6	0.054	19.0	106.0	0.9927	3.54	0.62	12.2	7	1	red
28	5.2	0.5	0.18	2.0	0.036	23.0	129.0	0.98949	3.36	0.77	13.4	7	1	white
29	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
30	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
31	5.3	0.165	0.24	1.1	0.051	25.0	105.0	0.9925	3.32	0.47	9.1	5	0	white
32	5.3	0.23	0.56	0.9	0.041	46.0	141.0	0.99119	3.16	0.62	9.7	5	0	white
33	5.3	0.3	0.3	1.2	0.029	25.0	93.0	0.98742	3.31	0.4	13.6	7	1	white
34	5.3	0.33	0.3	1.2	0.048	25.0	119.0	0.99045	3.32	0.62	11.3	6	0	white
35	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
36	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
37	5.3	0.4	0.25	3.9	0.031	45.0	130.0	0.99072	3.31	0.58	11.75	7	1	white
38	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.6	7	1	red
39	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.5666666666667	7	1	red
40	5.3	0.715	0.19	1.5	0.161	7.0	62.0	0.99395	3.62	0.61	11.0	5	0	red
41	5.4	0.22	0.29	1.2	0.045	69.0	152.0	0.99178	3.76	0.63	11.0	7	1	white
42	5.4	0.595	0.1	2.8	0.042	26.0	80.0	0.9932	3.36	0.38	9.3	5	0	white
43	5.4	0.74	0.09	1.7	0.089	16.0	26.0	0.99402	3.67	0.56	11.6	6	0	red
44	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
45	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
46	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
47	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
48	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.65	6	0	white
49	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.6333333333333	6	0	white
50	5.5	0.18	0.22	5.5	0.037	10.0	86.0	0.99156	3.46	0.44	12.2	5	0	white
51	5.5	0.24	0.45	1.7	0.046	22.0	113.0	0.99224	3.22	0.48	10.0	5	0	white
52	5.5	0.29	0.3	1.1	0.022	20.0	110.0	0.98869	3.34	0.38	12.8	7	1	white
53	5.5	0.31	0.29	3.0	0.027	16.0	102.0	0.99067	3.23	0.56	11.2	6	0	white
54	5.5	0.32	0.45	4.9	0.028	25.0	191.0	0.9922	3.51	0.49	11.5	7	1	white
55	5.5	0.35	0.35	1.1	0.045	14.0	167.0	0.992	3.34	0.68	9.9	6	0	white
56	5.5	0.375	0.38	1.7	0.036	17.0	98.0	0.99142	3.29	0.39	10.5	6	0	white
57	5.6	0.15	0.26	5.55	0.051	51.0	139.0	0.99336	3.47	0.5	11.0	6	0	white
58	5.6	0.15	0.31	5.3	0.038	8.0	79.0	0.9923	3.3	0.39	10.5	6	0	white
59	5.6	0.16	0.27	1.4	0.044	53.0	168.0	0.9918	3.28	0.37	10.1	6	0	white
60	5.6	0.175	0.29	0.8	0.043	20.0	67.0	0.99112	3.28	0.48	9.9	6	0	white
61	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
62	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
63	5.6	0.22	0.32	1.2	0.024	29.0	97.0	0.98823	3.2	0.46	13.05	7	1	white
64	5.6	0.26	0.18	1.4	0.034	18.0	135.0	0.99174	3.32	0.35	10.2	6	0	white
65	5.6	0.26	0.26	5.7	0.031	12.0	80.0	0.9923	3.25	0.38	10.8	5	0	white
66	5.6	0.26	0.5	11.4	0.029	25.0	93.0	0.99428	3.23	0.49	10.5	6	0	white
67	5.6	0.28	0.28	4.2	0.044	52.0	158.0	0.992	3.35	0.44	10.7	7	1	white
68	5.6	0.3	0.1	6.4	0.043	34.0	142.0	0.99382	3.14	0.48	9.8	5	0	white
69	5.6	0.35	0.14	5.0	0.046	48.0	198.0	0.9937	3.3	0.71	10.3	5	0	white
70	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
71	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
72	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
73	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
74	5.7	0.15	0.47	11.4	0.035	49.0	128.0	0.99456	3.03	0.34	10.5	8	1	white
75	5.7	0.18	0.26	2.2	0.023	21.0	95.0	0.9893	3.07	0.54	12.3	6	0	white
76	5.7	0.18	0.36	1.2	0.046	9.0	71.0	0.99199	3.7	0.68	10.9	7	1	white
77	5.7	0.2	0.3	2.5	0.046	38.0	125.0	0.99276	3.34	0.5	9.9	6	0	white
78	5.7	0.21	0.32	0.9	0.038	38.0	121.0	0.99074	3.24	0.46	10.6	6	0	white
79	5.7	0.21	0.37	4.5	0.04	58.0	140.0	0.99332	3.29	0.62	10.6	6	0	white
80	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
81	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
82	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
83	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
84	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
85	5.7	0.22	0.29	3.5	0.04	27.0	146.0	0.98999	3.17	0.36	12.1	6	0	white
86	5.7	0.23	0.28	9.65	0.025	26.0	121.0	0.9925	3.28	0.38	11.3	6	0	white
87	5.7	0.25	0.26	12.5	0.049	52.5	106.0	0.99691	3.08	0.45	9.4	6	0	white
88	5.7	0.25	0.26	12.5	0.049	52.5	120.0	0.99691	3.08	0.45	9.4	6	0	white
89	5.7	0.25	0.27	11.5	0.04	24.0	120.0	0.99411	3.33	0.31	10.8	6	0	white
90	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
91	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
92	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
93	5.7	0.27	0.32	1.2	0.046	20.0	155.0	0.9934	3.8	0.41	10.2	6	0	white
94	5.7	0.28	0.24	17.5	0.044	60.0	167.0	0.9989	3.31	0.44	9.4	5	0	white
95	5.7	0.32	0.18	1.4	0.029	26.0	104.0	0.9906	3.44	0.37	11.0	6	0	white
96	5.7	0.32	0.38	4.75	0.033	23.0	94.0	0.991	3.42	0.42	11.8	7	1	white
97	5.7	0.36	0.34	4.2	0.026	21.0	77.0	0.9907	3.41	0.45	11.9	6	0	white
98	5.8	0.14	0.15	6.1	0.042	27.0	123.0	0.99362	3.06	0.6	9.9	6	0	white
99	5.8	0.15	0.32	1.2	0.037	14.0	119.0	0.99137	3.19	0.5	10.2	6	0	white
100	5.8	0.17	0.34	1.8	0.045	96.0	170.0	0.99035	3.38	0.9	11.8	8	1	white

Rows: 1-100 | Columns: 14

Note

VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.

Next, we can initialize a LogisticRegression model:

from verticapy.machine_learning.vertica import LogisticRegression

model = LogisticRegression()

Now we can conveniently use the grid_search_cv() to search for the estimator using k-fold grid search.

from verticapy.machine_learning.model_selection import grid_search_cv

result = grid_search_cv(
    model,
    {
        "tol": [1e-2, 1e-4, 1e-6],
        "max_iter": [3, 10, 100],
        "solver": ["Newton", "BFGS"]
    },
    input_relation = data,
    X = [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density",
    ],
    y = "good",
    cv = 3,
)

		avg_score	avg_train_score	avg_time	score_std	score_train_std
1		0.187969356688602	0.18941845628866263	2.0343410968780518	0.0017104598472943728	0.0007828496228823299
2		0.18838743141945935	0.18916653066532566	2.2481016318003335	0.003303283273290816	0.001696875496825153
3		0.189111716688108	0.18887167488940068	2.2082297007242837	0.0045062363063493645	0.0022291566821730107
4		0.18926510801974036	0.18877130614133233	1.9629459381103516	0.001609820036387871	0.0008633665558033038
5		0.19074640064318701	0.18805136045798065	2.312570810317993	0.00399475425336779	0.0019533514137637428
6		0.19245829828761565	0.18707843920057698	2.095551093419393	0.002155255362110116	0.0010626924762442624
7		0.19250406556970534	0.18716341585441132	2.0909202893575034	0.0026235531770453006	0.0012257570213519706
8		0.19329354065473334	0.18680376734489967	2.345259189605713	0.0008787261064960804	0.00042192043979822014
9		0.19354039221124367	0.18671482616049165	2.034559965133667	0.0016867991282932212	0.0008229946631552568
10		0.19751014538820832	0.20269753594006265	24.989051898320515	0.0038706873693005483	0.0019115442554211155
11		0.21199505771400565	0.2137987736313113	6.440888325373332	0.003974964175746734	0.001650269801202728
12		0.21317081995071305	0.21085437722941133	6.859361410140991	0.001803382093482209	0.004946239033594465
13		0.21497402216561667	0.21320824583733566	4.170648495356242	0.0027427907674172137	0.0013408128007718254
14		0.21568198110241532	0.21250170110523733	5.532678206761678	0.004286763261332177	0.0020858421787152913
15		0.216454976707117	0.21245022407266667	4.182542244593303	0.0040491647662279934	0.0019302058259794465
16		0.2201556245446973	0.22713925184487935	2.9922386010487876	0.00267606658153402	0.000234490094657958
17		0.22697531479434266	0.22650071957081	2.862305482228597	0.011336719337324039	0.0012967521974187525
18		0.22899618755502135	0.22670225656058965	3.0409792264302573	0.006535852606597702	0.0005718926056397268

Rows: 1-18 | Columns: 6

Note

Grid Search in VerticaPy involves systematically evaluating different combinations of hyperparameters specified in a predefined grid. It iterates through all possible parameter combinations, training and evaluating models for each set. This exhaustive search helps identify the optimal hyperparameters that result in the best model performance. Grid Search is a powerful tool for fine-tuning machine learning models in VerticaPy to achieve better accuracy and generalization across various datasets.

See also

bayesian_search_cv() : Computes the k-fold bayesian search of an estimator.

randomized_search_cv() : Computes the K-Fold randomized search of an estimator.