Feedback

×

• Feedback
• Was this page helpful?*
  • Yes
  • No
• Do you have any additional feedback?
  • Yes
  • No
• Additional Feedback

verticapy.machine_learning.model_selection.hp_tuning.bayesian_search_cv

verticapy.machine_learning.model_selection.hp_tuning.bayesian_search_cv(estimator: VerticaModel, input_relation: Annotated[str | vDataFrame, ''], X: Annotated[str | list[str], 'STRING representing one column or a list of columns'], y: str, metric: str = 'auto', cv: int = 3, pos_label: Annotated[bool | float | str | timedelta | datetime, 'Python Scalar'] | None = None, cutoff: float = -1, param_grid: None | dict | list = None, random_nbins: int = 16, bayesian_nbins: int | None = None, random_grid: bool = False, lmax: int = 15, nrows: int = 100000, k_tops: int = 10, RFmodel_params: dict | None = None, print_info: bool = True, **kwargs) → TableSample

Computes the k-fold bayesian search of an estimator using a random forest model to estimate a probably optimal set of parameters.

estimator: object

Vertica estimator with a fit method.

input_relation: SQLRelation

Relation used to train the model.

X: SQLColumns

list of the predictor columns.

y: str

Response Column.

metric: str, optional

Metric used for the model evaluation.

• auto:

  logloss for classification & RMSE for regression.

For Classification

• accuracy:

  Accuracy.

  \[Accuracy = \frac{TP + TN}{TP + TN + FP + FN}\]

• auc:

  Area Under the Curve (ROC).

  \[AUC = \int_{0}^{1} TPR(FPR) \, dFPR\]

• ba:

  Balanced Accuracy.

  \[BA = \frac{TPR + TNR}{2}\]

• bm:

  Informedness

  \[BM = TPR + TNR - 1\]

• csi:

  Critical Success Index

  \[index = \frac{TP}{TP + FN + FP}\]

• f1:

  F1 Score .. math:

  F_1 Score = 2 \times 
  

rac{Precision times Recall}{Precision + Recall}

• fdr:

  False Discovery Rate

  \[FDR = 1 - PPV\]

• fm:

  Fowlkes-Mallows index

  \[FM = \sqrt{PPV * TPR}\]

• fnr:

  False Negative Rate

  \[FNR = \frac{FN}{FN + TP}\]

• for:

  False Omission Rate

  \[FOR = 1 - NPV\]

• fpr:

  False Positive Rate

  \[FPR = \frac{FP}{FP + TN}\]

• logloss:

  Log Loss

  \[Loss = -\frac{1}{N} \sum_{i=1}^{N} \left( y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right)\]

• lr+:

  Positive Likelihood Ratio.

  \[LR+ = \frac{TPR}{FPR}\]

• lr-:

  Negative Likelihood Ratio.

  \[LR- = \frac{FNR}{TNR}\]

• dor:

  Diagnostic Odds Ratio.

  \[DOR = \frac{TP \times TN}{FP \times FN}\]

• mcc:

  Matthews Correlation Coefficient

• mk:

  Markedness

  \[MK = PPV + NPV - 1\]

• npv:

  Negative Predictive Value

  \[NPV = \frac{TN}{TN + FN}\]

• prc_auc:

  Area Under the Curve (PRC)

  \[AUC = \int_{0}^{1} Precision(Recall) \, dRecall\]

• precision:

  Precision

  \[TP / (TP + FP)\]

• pt:

  Prevalence Threshold.

  \[\frac{\sqrt{FPR}}{\sqrt{TPR} + \sqrt{FPR}}\]

• recall:

  Recall.

  \[TP / (TP + FN)\]

• specificity:

  Specificity.

  \[TN / (TN + FP)\]

For Regression

• max:

  Max Error.

  \[ME = \max_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• mae:

  Mean Absolute Error.

  \[MAE = \frac{1}{n} \sum_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• median:

  Median Absolute Error.

  \[MedAE = \text{median}_{i=1}^{n} \left| y_i - \hat{y}_i \right|\]

• mse:

  Mean Squared Error.

  \[MSE = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2\]

• msle:

  Mean Squared Log Error.

  \[MSLE = \frac{1}{n} \sum_{i=1}^{n} (\log(1 + y_i) - \log(1 + \hat{y}_i))^2\]

• r2:

  R squared coefficient.

  \[R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}\]

• r2a:

  R2 adjusted

  \[\text{Adjusted } R^2 = 1 - \frac{(1 - R^2)(n - 1)}{n - k - 1}\]

• var:

  Explained Variance.

  \[VAR = 1 - \frac{Var(y - \hat{y})}{Var(y)}\]

• rmse:

  Root-mean-squared error

  \[RMSE = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2}\]

cv: int, optional

Number of folds.

pos_label: PythonScalar, optional

The main class to be considered as positive (classification only).

cutoff: float, optional

The model cutoff (classification only).

param_grid: dict | list, optional

Dictionary of the parameters to test. It can also be a list of the different combinations. If empty, a parameter grid is generated.

random_nbins: int, optional

Number of bins used to compute the different parameters categories in the random parameters generation.

bayesian_nbins: int, optional

Number of bins used to compute the different parameters categories in the bayesian table generation.

random_grid: bool, optional

If True, the rows used to find the optimal function are used randomnly. Otherwise, they are regularly spaced.

lmax: int, optional

Maximum length of each parameter list.

nrows: int, optional

Number of rows to use when performing the bayesian search.

k_tops: int, optional

When performing the bayesian search, the final stage is to retrain the top possible combinations. ‘k_tops’ represents the number of models to train at this stage in order to find the most efficient model.

RFmodel_params: dict, optional

dictionary of the random forest model parameters used to estimate a probably optimal set of parameters.

print_info: bool, optional

If True, prints the model information at each step.

TableSample

result of the bayesian search.

We import verticapy:

import verticapy as vp

Hint

By assigning an alias to verticapy, we mitigate the risk of code collisions with other libraries. This precaution is necessary because verticapy uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from verticapy are used as intended without interfering with functions from other libraries.

For this example, we will use the Wine Quality dataset.

import verticapy.datasets as vpd

data = vpd.load_winequality()

	123 fixed_acidity Numeric(8)	123 volatile_acidity Numeric(9)	123 citric_acid Numeric(8)	123 residual_sugar Numeric(9)	123 chlorides Float(22)	123 free_sulfur_dioxide Numeric(9)	123 total_sulfur_dioxide Numeric(9)	123 density Float(22)	123 pH Numeric(8)	123 sulphates Numeric(8)	123 alcohol Float(22)	123 quality Integer	123 good Integer	Abc color Varchar(20)
1	3.9	0.225	0.4	4.2	0.03	29.0	118.0	0.989	3.57	0.36	12.8	8	1	white
2	4.7	0.335	0.14	1.3	0.036	69.0	168.0	0.99212	3.47	0.46	10.5	5	0	white
3	4.7	0.455	0.18	1.9	0.036	33.0	106.0	0.98746	3.21	0.83	14.0	7	1	white
4	4.7	0.785	0.0	3.4	0.036	23.0	134.0	0.98981	3.53	0.92	13.8	6	0	white
5	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
6	4.9	0.345	0.34	1.0	0.068	32.0	143.0	0.99138	3.24	0.4	10.1	5	0	white
7	4.9	0.42	0.0	2.1	0.048	16.0	42.0	0.99154	3.71	0.74	14.0	7	1	red
8	5.0	0.27	0.4	1.2	0.076	42.0	124.0	0.99204	3.32	0.47	10.1	6	0	white
9	5.0	0.31	0.0	6.4	0.046	43.0	166.0	0.994	3.3	0.63	9.9	6	0	white
10	5.0	0.4	0.5	4.3	0.046	29.0	80.0	0.9902	3.49	0.66	13.6	6	0	red
11	5.0	0.44	0.04	18.6	0.039	38.0	128.0	0.9985	3.37	0.57	10.2	6	0	white
12	5.1	0.11	0.32	1.6	0.028	12.0	90.0	0.99008	3.57	0.52	12.2	6	0	white
13	5.1	0.14	0.25	0.7	0.039	15.0	89.0	0.9919	3.22	0.43	9.2	6	0	white
14	5.1	0.165	0.22	5.7	0.047	42.0	146.0	0.9934	3.18	0.55	9.9	6	0	white
15	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
16	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
17	5.1	0.33	0.22	1.6	0.027	18.0	89.0	0.9893	3.51	0.38	12.5	7	1	white
18	5.1	0.39	0.21	1.7	0.027	15.0	72.0	0.9894	3.5	0.45	12.5	6	0	white
19	5.2	0.2	0.27	3.2	0.047	16.0	93.0	0.99235	3.44	0.53	10.1	7	1	white
20	5.2	0.21	0.31	1.7	0.048	17.0	61.0	0.98953	3.24	0.37	12.0	7	1	white
21	5.2	0.22	0.46	6.2	0.066	41.0	187.0	0.99362	3.19	0.42	9.73333333333333	5	0	white
22	5.2	0.31	0.2	2.4	0.027	27.0	117.0	0.98886	3.56	0.45	13.0	7	1	white
23	5.2	0.32	0.25	1.8	0.103	13.0	50.0	0.9957	3.38	0.55	9.2	5	0	red
24	5.2	0.34	0.37	6.2	0.031	42.0	133.0	0.99076	3.25	0.41	12.5	6	0	white
25	5.2	0.36	0.02	1.6	0.031	24.0	104.0	0.9896	3.44	0.35	12.2	6	0	white
26	5.2	0.365	0.08	13.5	0.041	37.0	142.0	0.997	3.46	0.39	9.9	6	0	white
27	5.2	0.48	0.04	1.6	0.054	19.0	106.0	0.9927	3.54	0.62	12.2	7	1	red
28	5.2	0.5	0.18	2.0	0.036	23.0	129.0	0.98949	3.36	0.77	13.4	7	1	white
29	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
30	5.3	0.16	0.39	1.0	0.028	40.0	101.0	0.99156	3.57	0.59	10.6	6	0	white
31	5.3	0.165	0.24	1.1	0.051	25.0	105.0	0.9925	3.32	0.47	9.1	5	0	white
32	5.3	0.23	0.56	0.9	0.041	46.0	141.0	0.99119	3.16	0.62	9.7	5	0	white
33	5.3	0.3	0.3	1.2	0.029	25.0	93.0	0.98742	3.31	0.4	13.6	7	1	white
34	5.3	0.33	0.3	1.2	0.048	25.0	119.0	0.99045	3.32	0.62	11.3	6	0	white
35	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
36	5.3	0.36	0.27	6.3	0.028	40.0	132.0	0.99186	3.37	0.4	11.6	6	0	white
37	5.3	0.4	0.25	3.9	0.031	45.0	130.0	0.99072	3.31	0.58	11.75	7	1	white
38	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.6	7	1	red
39	5.3	0.47	0.11	2.2	0.048	16.0	89.0	0.99182	3.54	0.88	13.5666666666667	7	1	red
40	5.3	0.715	0.19	1.5	0.161	7.0	62.0	0.99395	3.62	0.61	11.0	5	0	red
41	5.4	0.22	0.29	1.2	0.045	69.0	152.0	0.99178	3.76	0.63	11.0	7	1	white
42	5.4	0.595	0.1	2.8	0.042	26.0	80.0	0.9932	3.36	0.38	9.3	5	0	white
43	5.4	0.74	0.09	1.7	0.089	16.0	26.0	0.99402	3.67	0.56	11.6	6	0	red
44	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
45	5.5	0.12	0.33	1.0	0.038	23.0	131.0	0.99164	3.25	0.45	9.8	5	0	white
46	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
47	5.5	0.14	0.27	4.6	0.029	22.0	104.0	0.9949	3.34	0.44	9.0	5	0	white
48	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.65	6	0	white
49	5.5	0.16	0.31	1.2	0.026	31.0	68.0	0.9898	3.33	0.44	11.6333333333333	6	0	white
50	5.5	0.18	0.22	5.5	0.037	10.0	86.0	0.99156	3.46	0.44	12.2	5	0	white
51	5.5	0.24	0.45	1.7	0.046	22.0	113.0	0.99224	3.22	0.48	10.0	5	0	white
52	5.5	0.29	0.3	1.1	0.022	20.0	110.0	0.98869	3.34	0.38	12.8	7	1	white
53	5.5	0.31	0.29	3.0	0.027	16.0	102.0	0.99067	3.23	0.56	11.2	6	0	white
54	5.5	0.32	0.45	4.9	0.028	25.0	191.0	0.9922	3.51	0.49	11.5	7	1	white
55	5.5	0.35	0.35	1.1	0.045	14.0	167.0	0.992	3.34	0.68	9.9	6	0	white
56	5.5	0.375	0.38	1.7	0.036	17.0	98.0	0.99142	3.29	0.39	10.5	6	0	white
57	5.6	0.15	0.26	5.55	0.051	51.0	139.0	0.99336	3.47	0.5	11.0	6	0	white
58	5.6	0.15	0.31	5.3	0.038	8.0	79.0	0.9923	3.3	0.39	10.5	6	0	white
59	5.6	0.16	0.27	1.4	0.044	53.0	168.0	0.9918	3.28	0.37	10.1	6	0	white
60	5.6	0.175	0.29	0.8	0.043	20.0	67.0	0.99112	3.28	0.48	9.9	6	0	white
61	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
62	5.6	0.185	0.19	7.1	0.048	36.0	110.0	0.99438	3.26	0.41	9.5	6	0	white
63	5.6	0.22	0.32	1.2	0.024	29.0	97.0	0.98823	3.2	0.46	13.05	7	1	white
64	5.6	0.26	0.18	1.4	0.034	18.0	135.0	0.99174	3.32	0.35	10.2	6	0	white
65	5.6	0.26	0.26	5.7	0.031	12.0	80.0	0.9923	3.25	0.38	10.8	5	0	white
66	5.6	0.26	0.5	11.4	0.029	25.0	93.0	0.99428	3.23	0.49	10.5	6	0	white
67	5.6	0.28	0.28	4.2	0.044	52.0	158.0	0.992	3.35	0.44	10.7	7	1	white
68	5.6	0.3	0.1	6.4	0.043	34.0	142.0	0.99382	3.14	0.48	9.8	5	0	white
69	5.6	0.35	0.14	5.0	0.046	48.0	198.0	0.9937	3.3	0.71	10.3	5	0	white
70	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
71	5.6	0.49	0.13	4.5	0.039	17.0	116.0	0.9907	3.42	0.9	13.7	7	1	white
72	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
73	5.6	0.66	0.0	2.2	0.087	3.0	11.0	0.99378	3.71	0.63	12.8	7	1	red
74	5.7	0.15	0.47	11.4	0.035	49.0	128.0	0.99456	3.03	0.34	10.5	8	1	white
75	5.7	0.18	0.26	2.2	0.023	21.0	95.0	0.9893	3.07	0.54	12.3	6	0	white
76	5.7	0.18	0.36	1.2	0.046	9.0	71.0	0.99199	3.7	0.68	10.9	7	1	white
77	5.7	0.2	0.3	2.5	0.046	38.0	125.0	0.99276	3.34	0.5	9.9	6	0	white
78	5.7	0.21	0.32	0.9	0.038	38.0	121.0	0.99074	3.24	0.46	10.6	6	0	white
79	5.7	0.21	0.37	4.5	0.04	58.0	140.0	0.99332	3.29	0.62	10.6	6	0	white
80	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
81	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
82	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
83	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
84	5.7	0.22	0.2	16.0	0.044	41.0	113.0	0.99862	3.22	0.46	8.9	6	0	white
85	5.7	0.22	0.29	3.5	0.04	27.0	146.0	0.98999	3.17	0.36	12.1	6	0	white
86	5.7	0.23	0.28	9.65	0.025	26.0	121.0	0.9925	3.28	0.38	11.3	6	0	white
87	5.7	0.25	0.26	12.5	0.049	52.5	106.0	0.99691	3.08	0.45	9.4	6	0	white
88	5.7	0.25	0.26	12.5	0.049	52.5	120.0	0.99691	3.08	0.45	9.4	6	0	white
89	5.7	0.25	0.27	11.5	0.04	24.0	120.0	0.99411	3.33	0.31	10.8	6	0	white
90	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
91	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
92	5.7	0.26	0.24	17.8	0.059	23.0	124.0	0.99773	3.3	0.5	10.1	5	0	white
93	5.7	0.27	0.32	1.2	0.046	20.0	155.0	0.9934	3.8	0.41	10.2	6	0	white
94	5.7	0.28	0.24	17.5	0.044	60.0	167.0	0.9989	3.31	0.44	9.4	5	0	white
95	5.7	0.32	0.18	1.4	0.029	26.0	104.0	0.9906	3.44	0.37	11.0	6	0	white
96	5.7	0.32	0.38	4.75	0.033	23.0	94.0	0.991	3.42	0.42	11.8	7	1	white
97	5.7	0.36	0.34	4.2	0.026	21.0	77.0	0.9907	3.41	0.45	11.9	6	0	white
98	5.8	0.14	0.15	6.1	0.042	27.0	123.0	0.99362	3.06	0.6	9.9	6	0	white
99	5.8	0.15	0.32	1.2	0.037	14.0	119.0	0.99137	3.19	0.5	10.2	6	0	white
100	5.8	0.17	0.34	1.8	0.045	96.0	170.0	0.99035	3.38	0.9	11.8	8	1	white

Rows: 1-100 | Columns: 14

Note

VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.

Next, we can initialize a LogisticRegression model:

from verticapy.machine_learning.vertica import LogisticRegression

model = LogisticRegression()

Now we can conveniently use the bayesian_search_cv() function to find an optimal set of parameters estimator.

from verticapy.machine_learning.model_selection import bayesian_search_cv

result = bayesian_search_cv(
    model,
    input_relation = data,
    X = [
        "fixed_acidity",
        "volatile_acidity",
        "citric_acid",
        "residual_sugar",
        "chlorides",
        "density",
    ],
    y = "good",
    cv = 3,
)

		avg_score	avg_train_score	avg_time	score_std	score_train_std
1		0.20703241936921768	0.206178499236787	27.9562615553538	0.0034771478877264604	0.0016549858510009443
2		0.21363957221654697	0.214258181987338	3.8594278494517007	0.00405424931081192	0.0017153510086339277
3		0.213684601973023	0.21378231308508067	3.9229041735331216	0.00048494876920781205	0.0002870845235435017
4		0.21421319760967936	0.21366991400344668	4.0780251026153564	0.003011231181720533	0.0015761895339180953
5		0.216878417176065	0.2122496358171143	4.031023263931274	0.0032725912416497974	0.0016522984211014026
6		0.28163452853357135	0.285016198697723	1.8422833283742268	0.0006963401985030411	0.0005747194005340293
7		0.2840327651298207	0.28465732891806866	1.9104742209116619	0.002260756203409748	0.0011296162376252309
8		0.284066437432585	0.28499135507846335	1.9960723718007405	0.0008662167887728358	0.0004954236274347878
9		0.285320488070419	0.28467209375439334	1.9603406588236492	0.0016804960091023494	0.00044352977296118417
10		0.28557191721608	0.284683738523249	1.5813016096750896	0.0025460433464253246	0.0006687146817120286
11		0.28760491813867134	0.2845857618733227	1.743993918100993	0.0020044617350943443	0.0007214873368928489
12		0.29523065308050933	0.29581302213151633	1.8195548852284749	0.0008014887079443658	0.0002667603255413938
13		0.30094946343310636	0.30090330657867265	1.5251398086547852	2.690228652368777e-05	6.394853432370082e-05
14		0.301029995663981	0.301029995663981	1.520217736562093	0.0	0.0
15		0.301029995663981	0.301029995663981	1.5820024808247883	0.0	0.0
16		0.301029995663981	0.301029995663981	1.5456359386444092	0.0	0.0
17		0.301029995663981	0.301029995663981	1.548525094985962	0.0	0.0
18		0.301029995663981	0.301029995663981	1.5311799844106038	0.0	0.0
19		0.301029995663981	0.301029995663981	1.5185945828755696	0.0	0.0
20		0.301029995663981	0.301029995663981	1.5129992961883545	0.0	0.0
21		0.301029995663981	0.301029995663981	1.5183689594268799	0.0	0.0
22		0.301029995663981	0.301029995663981	1.4661417802174885	0.0	0.0
23		0.301029995663981	0.301029995663981	1.505034605662028	0.0	0.0
24		0.301029995663981	0.301029995663981	1.6606460412343342	0.0	0.0
25		0.301029995663981	0.301029995663981	1.564717690149943	0.0	0.0

Rows: 1-25 | Columns: 6

Note

In VerticaPy, Bayesian optimization utilizes the RandomForest model instead of Gaussian processes. The optimization process involves iteratively selecting and evaluating sets of parameters, with the RandomForest model predicting the performance of different parameter configurations. This approach allows VerticaPy to efficiently navigate the parameter space and identify optimal settings for machine learning models, enhancing their overall performance.

See also

grid_search_cv() : Computes the k-fold grid search of an estimator.

randomized_search_cv() : Computes the K-Fold randomized search of an estimator.