cross_validate¶

In [ ]:

cross_validate(estimator,
               input_relation: (str, vDataFrame),
               X: list,
               y: str,
               metric: (str, list) = "all",
               cv: int = 3,
               pos_label: (int, float, str) = None,
               cutoff: float = -1,
               show_time: bool = True,
               training_score: bool = False,)

Computes the k-fold cross validation of an estimator.

Parameters¶

Name	Type	Optional	Description
estimator	object	❌	Vertica estimator having a fit method.
input_relation	str / vDataFrame	❌	Input Relation.
X	list	❌	List of the predictor columns.
y	str	❌	Response Column.
metric	str / list	✓	Metric used to do the model evaluation. It can also be a list of metrics. all : The model will compute all the possible metrics. For Classification: accuracy : Accuracy auc : Area Under the Curve (ROC) best_cutoff : Cutoff which optimised the ROC Curve prediction. bm : Informedness = tpr + tnr - 1 csi : Critical Success Index = tp / (tp + fn + fp) f1 : F1 Score logloss : Log Loss mcc : Matthews Correlation Coefficient mk : Markedness = ppv + npv - 1 npv : Negative Predictive Value = tn / (tn + fn) prc_auc : Area Under the Curve (PRC) precision : Precision = tp / (tp + fp) recall : Recall = tp / (tp + fn) specificity : Specificity = tn / (tn + fp) For Regression: max : Max Error mae : Mean Absolute Error median : Median Absolute Error mse : Mean Squared Error msle : Mean Squared Log Error r2 : R-squared coefficient r2a : R2 adjusted rmse : Root Mean Squared Error var : Explained Variance
cv	int	✓	Number of folds.
pos_label	int / float / str	✓	The main class to be considered as positive (classification only).
cutoff	float	✓	The model cutoff (classification only).
show_time	bool	✓	If set to True, the time and the average time will be added to the report.
training_score	bool	✓	If set to True, the training score will be computed with the validation score.

Returns¶

tablesample : An object containing the result. For more information, see utilities.tablesample.

Example¶

In [3]:

from verticapy.learn.linear_model import LogisticRegression
model = LogisticRegression(name = "public.LR_titanic",
                           tol = 1e-4,
                           max_iter = 100, 
                           solver = 'Newton')

from verticapy.learn.model_selection import cross_validate
cross_validate(model, 
               input_relation = "public.titanic_clean", 
               X = ["age", "fare", "parch", "sex", "boat"], 
               y = "survived", 
               cv = 3)

Out[3]:

	auc	prc_auc	accuracy	log_loss	precision	recall	f1_score	mcc	informedness	markedness	csi	time
1-fold	0.9870639270592511	0.9754993614768523	0.9736842105263158	0.0465068331622295	0.9701492537313433	0.948905109489051	0.9669839820620577	0.9400702013302654	0.9346702340442112	0.9455013664073997	0.9219858156028369	1.5210838317871094
2-fold	0.9965034965034969	0.9940433335943306	0.9822335025380711	0.0268999625169969	0.9857142857142858	0.965034965034965	0.9783472133247918	0.9615375997716146	0.9570668375449252	0.9660292463442071	0.9517241379310345	1.1832020282745361
3-fold	0.9942676076623937	0.988089831718706	0.9772727272727273	0.03480559637082	0.9765625	0.9541984732824428	0.971132889528074	0.948509261621695	0.9428777185654615	0.9541744402985075	0.9328358208955224	1.250831127166748
avg	0.9926116770750473	0.985877508929963	0.977730146779038	0.03607079735001546	0.9774753464818764	0.9560461826021529	0.9721546949716412	0.9500390209078583	0.9448715967181993	0.9552350176833714	0.9355152581431313	1.318372329076131
std	0.004932843744308882	0.009467866949035914	0.004292962022791574	0.009864476380580918	0.007822564904307982	0.008222139762897526	0.005750114864589401	0.010815147782860489	0.011330649789752643	0.010304954243807117	0.015049135911966823	0.1787802806596699

Rows: 1-5 | Columns: 13

In [5]:

# Adding the train result

res_test, res_train = cross_validate(
                           model, 
                           input_relation = "public.titanic_clean", 
                           X = ["age", "fare", "parch", "sex", "boat"], 
                           y = "survived", 
                           cv = 3,
                           metric = "auc",
                           training_score = True)
display(res_test)

	auc	time
1-fold	0.994964422550629	1.060049057006836
2-fold	0.9865709234457875	1.4258790016174316
3-fold	0.994085467147849	1.1313872337341309
avg	0.9918736043814218	1.205771764119466
std	0.004613237418437735	0.1939269988009226

Rows: 1-5 | Columns: 3

In [6]:

display(res_train)

	auc	time
1-fold	0.9905014494541426	1.060049057006836
2-fold	0.9951098242419082	1.4258790016174316
3-fold	0.990764884832683	1.1313872337341309
avg	0.9921253861762446	1.205771764119466
std	0.002587953337195826	0.1939269988009226

Rows: 1-5 | Columns: 3