validation_curve¶

In [ ]:

validation_curve(estimator,
                 param_name: str,
                 param_range: list,
                 input_relation: (str, vDataFrame),
                 X: list,
                 y: str,
                 metric: str = "auto",
                 cv: int = 3,
                 pos_label: (int, float, str) = None,
                 cutoff: float = -1,
                 std_coeff: float = 1,
                 ax=None,
                 **style_kwds,)

Draws the validation curve.

Parameters¶

Name	Type	Optional	Description
estimator	object	❌	Vertica estimator having a fit method.
param_name	str	❌	Parameter name.
param_range	list	❌	Parameter Range.
input_relation	str / vDataFrame	❌	Input Relation.
X	list	❌	List of the predictor columns.
y	str	❌	Response Column.
metric	str / list	✓	Metric used to do the model evaluation. It can also be a list of metrics. auto : logloss for classification & rmse for regression. For Classification: accuracy : Accuracy auc : Area Under the Curve (ROC) bm : Informedness = tpr + tnr - 1 csi : Critical Success Index = tp / (tp + fn + fp) f1 : F1 Score logloss : Log Loss mcc : Matthews Correlation Coefficient mk : Markedness = ppv + npv - 1 npv : Negative Predictive Value = tn / (tn + fn) prc_auc : Area Under the Curve (PRC) precision : Precision = tp / (tp + fp) recall : Recall = tp / (tp + fn) specificity : Specificity = tn / (tn + fp) For Regression: max : Max Error mae : Mean Absolute Error median : Median Absolute Error mse : Mean Squared Error msle : Mean Squared Log Error r2 : R-squared coefficient r2a : R2 adjusted rmse : Root Mean Squared Error var : Explained Variance
cv	int	✓	Number of folds.
pos_label	int / float / str	✓	The main class to be considered as positive (classification only).
cutoff	float	✓	The model cutoff (classification only).
std_coeff	float	✓	Value of the standard deviation coefficient used to compute the area plot around each score.
ax	Matplotlib axes object	✓	The axes to plot on.
**style_kwds	any	✓	Any optional parameter to pass to the Matplotlib functions.

Returns¶

tablesample : An object containing the result. For more information, see utilities.tablesample.

Example¶

In [5]:

from verticapy.learn.ensemble import RandomForestClassifier
model = RandomForestClassifier(name = "public.RF_titanic")

from verticapy.learn.model_selection import validation_curve
validation_curve(model,
                 "max_depth",
                 [1, 5, 8, 10, 20, 30, 40, 50, 60],
                 input_relation = "public.titanic_clean", 
                 X = ["age", "fare", "parch",],
                 y = "survived", 
                 cv = 3,
                 metric = "auc",)

Out[5]:

	max_depth	training_score_lower	training_score	training_score_upper	test_score_lower	test_score	test_score_upper
1	1	0.658263707229827	0.6818003162125857	0.7053369251953445	0.6108143853078426	0.6403709719710954	0.6699275586343482
2	5	0.7613470513705837	0.7693006552836843	0.7772542591967849	0.6772073386077834	0.6923113507202922	0.7074153628328009
3	8	0.8200436648925985	0.824515358198933	0.8289870515052674	0.6661442514783809	0.6751330890722373	0.6841219266660938
4	10	0.8347311333295033	0.8449917787101974	0.8552524240908914	0.633877072464402	0.6471877213471875	0.660498370229973
5	20	0.8394748590298599	0.8480323191634934	0.856589779297127	0.6633939142695147	0.6793629422712388	0.695331970272963
6	30	0.8433729698394641	0.8472043207871567	0.8510356717348493	0.6363774509873922	0.6560663367021203	0.6757552224168485
7	40	0.853652537930711	0.8631169822239596	0.8725814265172082	0.6385935139270961	0.6608057186675838	0.6830179234080714
8	50	0.8419220595748879	0.8488815414760171	0.8558410233771464	0.6549637905075147	0.6595493317654872	0.6641348730234597
9	60	0.8505064353418901	0.8553410543122557	0.8601756732826213	0.6533425728521399	0.6589127044603263	0.6644828360685128

Rows: 1-9 | Columns: 7