{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Classification" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Classifications are ML algorithms used to predict categorical response columns. For predicting more than two categories, these are called 'Multiclass Classifications'. Examples of classification are predicting the flower species using specific characteristics or predicting whether Telco customers will churn.\n", "\n", "To understand how to create a classification model, let's predict the species of flowers with the Iris dataset. \n", "\n", "We'll start by importing the Random Forest Classifier." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from verticapy.learn.ensemble import RandomForestClassifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we'll create a model object. Vertica has its own model management system, so we just need to choose a model name. This name must include the model's schema, which is 'public' by default." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "model = RandomForestClassifier(\"RF_Iris\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's use the iris dataset." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from verticapy.datasets import load_iris\n", "iris = load_iris()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that the data is loaded, we can fit the model." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\n", "\n", "===========\n", "call_string\n", "===========\n", "SELECT rf_classifier('public.RF_Iris', '\"public\".\"_verticapy_tmp_view_dbadmin_178273_7491533984_\"', '\"species\"', '\"PetalLengthCm\", \"SepalLengthCm\"' USING PARAMETERS exclude_columns='', ntree=10, mtry=1, sampling_size=0.632, max_depth=5, max_breadth=1000000000, min_leaf_size=1, min_info_gain=0, nbins=32);\n", "\n", "=======\n", "details\n", "=======\n", " predictor | type \n", "-------------+----------------\n", "petallengthcm|float or numeric\n", "sepallengthcm|float or numeric\n", "\n", "\n", "===============\n", "Additional Info\n", "===============\n", " Name |Value\n", "------------------+-----\n", " tree_count | 10 \n", "rejected_row_count| 0 \n", "accepted_row_count| 150 " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.fit(iris, [\"PetalLengthCm\", \"SepalLengthCm\"], \"Species\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have many metrics to evaluate the model." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Iris-setosa
Iris-versicolor
Iris-virginica
auc1.00.99600000000000020.9954000000000001
prc_auc1.00.99202633839390870.9918601369253075
accuracy1.00.980.9733333333333334
log_loss0.01434337980566460.04762126678181530.0466894193311429
precision1.00.96078431372549020.96
recall1.00.980.96
f1_score1.00.97029702970297020.96
mcc1.00.95533029444245140.94
informedness1.00.960.94
markedness1.00.95068330362448020.94
csi1.00.94230769230769230.9230769230769231
cutoff0.80.44020.4927
Rows: 1-12 | Columns: 4
" ], "text/plain": [ " Iris-setosa Iris-versicolor Iris-virginica \n", "auc 1.0 0.9960000000000002 0.9954000000000001 \n", "prc_auc 1.0 0.9920263383939087 0.9918601369253075 \n", "accuracy 1.0 0.98 0.9733333333333334 \n", "log_loss 0.0143433798056646 0.0476212667818153 0.0466894193311429 \n", "precision 1.0 0.9607843137254902 0.96 \n", "recall 1.0 0.98 0.96 \n", "f1_score 1.0 0.9702970297029702 0.96 \n", "mcc 1.0 0.9553302944424514 0.94 \n", "informedness 1.0 0.96 0.94 \n", "markedness 1.0 0.9506833036244802 0.94 \n", "csi 1.0 0.9423076923076923 0.9230769230769231 \n", "cutoff 0.8 0.4402 0.4927 \n", "Rows: 1-12 | Columns: 4" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.report()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can add the predictions to your dataset." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
123
Id
Int
123
PetalLengthCm
Numeric(6,3)
123
PetalWidthCm
Numeric(6,3)
123
SepalLengthCm
Numeric(6,3)
123
SepalWidthCm
Numeric(6,3)
Abc
Species
Varchar(30)
Abc
prediction
Varchar(128)
111.40.25.13.5Iris-setosaIris-setosa
221.40.24.93.0Iris-setosaIris-setosa
331.30.24.73.2Iris-setosaIris-setosa
441.50.24.63.1Iris-setosaIris-setosa
551.40.25.03.6Iris-setosaIris-setosa
661.70.45.43.9Iris-setosaIris-setosa
771.40.34.63.4Iris-setosaIris-setosa
881.50.25.03.4Iris-setosaIris-setosa
991.40.24.42.9Iris-setosaIris-setosa
10101.50.14.93.1Iris-setosaIris-setosa
11111.50.25.43.7Iris-setosaIris-setosa
12121.60.24.83.4Iris-setosaIris-setosa
13131.40.14.83.0Iris-setosaIris-setosa
14141.10.14.33.0Iris-setosaIris-setosa
15151.20.25.84.0Iris-setosaIris-setosa
16161.50.45.74.4Iris-setosaIris-setosa
17171.30.45.43.9Iris-setosaIris-setosa
18181.40.35.13.5Iris-setosaIris-setosa
19191.70.35.73.8Iris-setosaIris-setosa
20201.50.35.13.8Iris-setosaIris-setosa
21211.70.25.43.4Iris-setosaIris-setosa
22221.50.45.13.7Iris-setosaIris-setosa
23231.00.24.63.6Iris-setosaIris-setosa
24241.70.55.13.3Iris-setosaIris-setosa
25251.90.24.83.4Iris-setosaIris-setosa
26261.60.25.03.0Iris-setosaIris-setosa
27271.60.45.03.4Iris-setosaIris-setosa
28281.50.25.23.5Iris-setosaIris-setosa
29291.40.25.23.4Iris-setosaIris-setosa
30301.60.24.73.2Iris-setosaIris-setosa
31311.60.24.83.1Iris-setosaIris-setosa
32321.50.45.43.4Iris-setosaIris-setosa
33331.50.15.24.1Iris-setosaIris-setosa
34341.40.25.54.2Iris-setosaIris-setosa
35351.50.14.93.1Iris-setosaIris-setosa
36361.20.25.03.2Iris-setosaIris-setosa
37371.30.25.53.5Iris-setosaIris-setosa
38381.50.14.93.1Iris-setosaIris-setosa
39391.30.24.43.0Iris-setosaIris-setosa
40401.50.25.13.4Iris-setosaIris-setosa
41411.30.35.03.5Iris-setosaIris-setosa
42421.30.34.52.3Iris-setosaIris-setosa
43431.30.24.43.2Iris-setosaIris-setosa
44441.60.65.03.5Iris-setosaIris-setosa
45451.90.45.13.8Iris-setosaIris-setosa
46461.40.34.83.0Iris-setosaIris-setosa
47471.60.25.13.8Iris-setosaIris-setosa
48481.40.24.63.2Iris-setosaIris-setosa
49491.50.25.33.7Iris-setosaIris-setosa
50501.40.25.03.3Iris-setosaIris-setosa
51514.71.47.03.2Iris-versicolorIris-versicolor
52524.51.56.43.2Iris-versicolorIris-versicolor
53534.91.56.93.1Iris-versicolorIris-versicolor
54544.01.35.52.3Iris-versicolorIris-versicolor
55554.61.56.52.8Iris-versicolorIris-versicolor
56564.51.35.72.8Iris-versicolorIris-versicolor
57574.71.66.33.3Iris-versicolorIris-versicolor
58583.31.04.92.4Iris-versicolorIris-versicolor
59594.61.36.62.9Iris-versicolorIris-versicolor
60603.91.45.22.7Iris-versicolorIris-versicolor
61613.51.05.02.0Iris-versicolorIris-versicolor
62624.21.55.93.0Iris-versicolorIris-versicolor
63634.01.06.02.2Iris-versicolorIris-versicolor
64644.71.46.12.9Iris-versicolorIris-versicolor
65653.61.35.62.9Iris-versicolorIris-versicolor
66664.41.46.73.1Iris-versicolorIris-versicolor
67674.51.55.63.0Iris-versicolorIris-versicolor
68684.11.05.82.7Iris-versicolorIris-versicolor
69694.51.56.22.2Iris-versicolorIris-versicolor
70703.91.15.62.5Iris-versicolorIris-versicolor
71714.81.85.93.2Iris-versicolorIris-versicolor
72724.01.36.12.8Iris-versicolorIris-versicolor
73734.91.56.32.5Iris-versicolorIris-virginica
74744.71.26.12.8Iris-versicolorIris-versicolor
75754.31.36.42.9Iris-versicolorIris-versicolor
76764.41.46.63.0Iris-versicolorIris-versicolor
77774.81.46.82.8Iris-versicolorIris-versicolor
78785.01.76.73.0Iris-versicolorIris-versicolor
79794.51.56.02.9Iris-versicolorIris-versicolor
80803.51.05.72.6Iris-versicolorIris-versicolor
81813.81.15.52.4Iris-versicolorIris-versicolor
82823.71.05.52.4Iris-versicolorIris-versicolor
83833.91.25.82.7Iris-versicolorIris-versicolor
84845.11.66.02.7Iris-versicolorIris-virginica
85854.51.55.43.0Iris-versicolorIris-versicolor
86864.51.66.03.4Iris-versicolorIris-versicolor
87874.71.56.73.1Iris-versicolorIris-versicolor
88884.41.36.32.3Iris-versicolorIris-versicolor
89894.11.35.63.0Iris-versicolorIris-versicolor
90904.01.35.52.5Iris-versicolorIris-versicolor
91914.41.25.52.6Iris-versicolorIris-versicolor
92924.61.46.13.0Iris-versicolorIris-versicolor
93934.01.25.82.6Iris-versicolorIris-versicolor
94943.31.05.02.3Iris-versicolorIris-versicolor
95954.21.35.62.7Iris-versicolorIris-versicolor
96964.21.25.73.0Iris-versicolorIris-versicolor
97974.21.35.72.9Iris-versicolorIris-versicolor
98984.31.36.22.9Iris-versicolorIris-versicolor
99993.01.15.12.5Iris-versicolorIris-versicolor
1001004.11.35.72.8Iris-versicolorIris-versicolor
Rows: 1-100 | Columns: 7
" ], "text/plain": [ " Id PetalLengthCm PetalWidthCm SepalLengthCm \\\\\n", "1 1 1.4 0.2 5.1 \\\\\n", "2 2 1.4 0.2 4.9 \\\\\n", "3 3 1.3 0.2 4.7 \\\\\n", "4 4 1.5 0.2 4.6 \\\\\n", "5 5 1.4 0.2 5.0 \\\\\n", "6 6 1.7 0.4 5.4 \\\\\n", "7 7 1.4 0.3 4.6 \\\\\n", "8 8 1.5 0.2 5.0 \\\\\n", "9 9 1.4 0.2 4.4 \\\\\n", "10 10 1.5 0.1 4.9 \\\\\n", "11 11 1.5 0.2 5.4 \\\\\n", "12 12 1.6 0.2 4.8 \\\\\n", "13 13 1.4 0.1 4.8 \\\\\n", "14 14 1.1 0.1 4.3 \\\\\n", "15 15 1.2 0.2 5.8 \\\\\n", "16 16 1.5 0.4 5.7 \\\\\n", "17 17 1.3 0.4 5.4 \\\\\n", "18 18 1.4 0.3 5.1 \\\\\n", "19 19 1.7 0.3 5.7 \\\\\n", "20 20 1.5 0.3 5.1 \\\\\n", "21 21 1.7 0.2 5.4 \\\\\n", "22 22 1.5 0.4 5.1 \\\\\n", "23 23 1.0 0.2 4.6 \\\\\n", "24 24 1.7 0.5 5.1 \\\\\n", "25 25 1.9 0.2 4.8 \\\\\n", "26 26 1.6 0.2 5.0 \\\\\n", "27 27 1.6 0.4 5.0 \\\\\n", "28 28 1.5 0.2 5.2 \\\\\n", "29 29 1.4 0.2 5.2 \\\\\n", "30 30 1.6 0.2 4.7 \\\\\n", "31 31 1.6 0.2 4.8 \\\\\n", "32 32 1.5 0.4 5.4 \\\\\n", "33 33 1.5 0.1 5.2 \\\\\n", "34 34 1.4 0.2 5.5 \\\\\n", "35 35 1.5 0.1 4.9 \\\\\n", "36 36 1.2 0.2 5.0 \\\\\n", "37 37 1.3 0.2 5.5 \\\\\n", "38 38 1.5 0.1 4.9 \\\\\n", "39 39 1.3 0.2 4.4 \\\\\n", "40 40 1.5 0.2 5.1 \\\\\n", "41 41 1.3 0.3 5.0 \\\\\n", "42 42 1.3 0.3 4.5 \\\\\n", "43 43 1.3 0.2 4.4 \\\\\n", "44 44 1.6 0.6 5.0 \\\\\n", "45 45 1.9 0.4 5.1 \\\\\n", "46 46 1.4 0.3 4.8 \\\\\n", "47 47 1.6 0.2 5.1 \\\\\n", "48 48 1.4 0.2 4.6 \\\\\n", "49 49 1.5 0.2 5.3 \\\\\n", "50 50 1.4 0.2 5.0 \\\\\n", "51 51 4.7 1.4 7.0 \\\\\n", "52 52 4.5 1.5 6.4 \\\\\n", "53 53 4.9 1.5 6.9 \\\\\n", "54 54 4.0 1.3 5.5 \\\\\n", "55 55 4.6 1.5 6.5 \\\\\n", "56 56 4.5 1.3 5.7 \\\\\n", "57 57 4.7 1.6 6.3 \\\\\n", "58 58 3.3 1.0 4.9 \\\\\n", "59 59 4.6 1.3 6.6 \\\\\n", "60 60 3.9 1.4 5.2 \\\\\n", "61 61 3.5 1.0 5.0 \\\\\n", "62 62 4.2 1.5 5.9 \\\\\n", "63 63 4.0 1.0 6.0 \\\\\n", "64 64 4.7 1.4 6.1 \\\\\n", "65 65 3.6 1.3 5.6 \\\\\n", "66 66 4.4 1.4 6.7 \\\\\n", "67 67 4.5 1.5 5.6 \\\\\n", "68 68 4.1 1.0 5.8 \\\\\n", "69 69 4.5 1.5 6.2 \\\\\n", "70 70 3.9 1.1 5.6 \\\\\n", "71 71 4.8 1.8 5.9 \\\\\n", "72 72 4.0 1.3 6.1 \\\\\n", "73 73 4.9 1.5 6.3 \\\\\n", "74 74 4.7 1.2 6.1 \\\\\n", "75 75 4.3 1.3 6.4 \\\\\n", "76 76 4.4 1.4 6.6 \\\\\n", "77 77 4.8 1.4 6.8 \\\\\n", "78 78 5.0 1.7 6.7 \\\\\n", "79 79 4.5 1.5 6.0 \\\\\n", "80 80 3.5 1.0 5.7 \\\\\n", "81 81 3.8 1.1 5.5 \\\\\n", "82 82 3.7 1.0 5.5 \\\\\n", "83 83 3.9 1.2 5.8 \\\\\n", "84 84 5.1 1.6 6.0 \\\\\n", "85 85 4.5 1.5 5.4 \\\\\n", "86 86 4.5 1.6 6.0 \\\\\n", "87 87 4.7 1.5 6.7 \\\\\n", "88 88 4.4 1.3 6.3 \\\\\n", "89 89 4.1 1.3 5.6 \\\\\n", "90 90 4.0 1.3 5.5 \\\\\n", "91 91 4.4 1.2 5.5 \\\\\n", "92 92 4.6 1.4 6.1 \\\\\n", "93 93 4.0 1.2 5.8 \\\\\n", "94 94 3.3 1.0 5.0 \\\\\n", "95 95 4.2 1.3 5.6 \\\\\n", "96 96 4.2 1.2 5.7 \\\\\n", "97 97 4.2 1.3 5.7 \\\\\n", "98 98 4.3 1.3 6.2 \\\\\n", "99 99 3.0 1.1 5.1 \\\\\n", "100 100 4.1 1.3 5.7 \\\\\n", " SepalWidthCm Species prediction \n", "1 3.5 Iris-setosa Iris-setosa \n", "2 3.0 Iris-setosa Iris-setosa \n", "3 3.2 Iris-setosa Iris-setosa \n", "4 3.1 Iris-setosa Iris-setosa \n", "5 3.6 Iris-setosa Iris-setosa \n", "6 3.9 Iris-setosa Iris-setosa \n", "7 3.4 Iris-setosa Iris-setosa \n", "8 3.4 Iris-setosa Iris-setosa \n", "9 2.9 Iris-setosa Iris-setosa \n", "10 3.1 Iris-setosa Iris-setosa \n", "11 3.7 Iris-setosa Iris-setosa \n", "12 3.4 Iris-setosa Iris-setosa \n", "13 3.0 Iris-setosa Iris-setosa \n", "14 3.0 Iris-setosa Iris-setosa \n", "15 4.0 Iris-setosa Iris-setosa \n", "16 4.4 Iris-setosa Iris-setosa \n", "17 3.9 Iris-setosa Iris-setosa \n", "18 3.5 Iris-setosa Iris-setosa \n", "19 3.8 Iris-setosa Iris-setosa \n", "20 3.8 Iris-setosa Iris-setosa \n", "21 3.4 Iris-setosa Iris-setosa \n", "22 3.7 Iris-setosa Iris-setosa \n", "23 3.6 Iris-setosa Iris-setosa \n", "24 3.3 Iris-setosa Iris-setosa \n", "25 3.4 Iris-setosa Iris-setosa \n", "26 3.0 Iris-setosa Iris-setosa \n", "27 3.4 Iris-setosa Iris-setosa \n", "28 3.5 Iris-setosa Iris-setosa \n", "29 3.4 Iris-setosa Iris-setosa \n", "30 3.2 Iris-setosa Iris-setosa \n", "31 3.1 Iris-setosa Iris-setosa \n", "32 3.4 Iris-setosa Iris-setosa \n", "33 4.1 Iris-setosa Iris-setosa \n", "34 4.2 Iris-setosa Iris-setosa \n", "35 3.1 Iris-setosa Iris-setosa \n", "36 3.2 Iris-setosa Iris-setosa \n", "37 3.5 Iris-setosa Iris-setosa \n", "38 3.1 Iris-setosa Iris-setosa \n", "39 3.0 Iris-setosa Iris-setosa \n", "40 3.4 Iris-setosa Iris-setosa \n", "41 3.5 Iris-setosa Iris-setosa \n", "42 2.3 Iris-setosa Iris-setosa \n", "43 3.2 Iris-setosa Iris-setosa \n", "44 3.5 Iris-setosa Iris-setosa \n", "45 3.8 Iris-setosa Iris-setosa \n", "46 3.0 Iris-setosa Iris-setosa \n", "47 3.8 Iris-setosa Iris-setosa \n", "48 3.2 Iris-setosa Iris-setosa \n", "49 3.7 Iris-setosa Iris-setosa \n", "50 3.3 Iris-setosa Iris-setosa \n", "51 3.2 Iris-versicolor Iris-versicolor \n", "52 3.2 Iris-versicolor Iris-versicolor \n", "53 3.1 Iris-versicolor Iris-versicolor \n", "54 2.3 Iris-versicolor Iris-versicolor \n", "55 2.8 Iris-versicolor Iris-versicolor \n", "56 2.8 Iris-versicolor Iris-versicolor \n", "57 3.3 Iris-versicolor Iris-versicolor \n", "58 2.4 Iris-versicolor Iris-versicolor \n", "59 2.9 Iris-versicolor Iris-versicolor \n", "60 2.7 Iris-versicolor Iris-versicolor \n", "61 2.0 Iris-versicolor Iris-versicolor \n", "62 3.0 Iris-versicolor Iris-versicolor \n", "63 2.2 Iris-versicolor Iris-versicolor \n", "64 2.9 Iris-versicolor Iris-versicolor \n", "65 2.9 Iris-versicolor Iris-versicolor \n", "66 3.1 Iris-versicolor Iris-versicolor \n", "67 3.0 Iris-versicolor Iris-versicolor \n", "68 2.7 Iris-versicolor Iris-versicolor \n", "69 2.2 Iris-versicolor Iris-versicolor \n", "70 2.5 Iris-versicolor Iris-versicolor \n", "71 3.2 Iris-versicolor Iris-versicolor \n", "72 2.8 Iris-versicolor Iris-versicolor \n", "73 2.5 Iris-versicolor Iris-virginica \n", "74 2.8 Iris-versicolor Iris-versicolor \n", "75 2.9 Iris-versicolor Iris-versicolor \n", "76 3.0 Iris-versicolor Iris-versicolor \n", "77 2.8 Iris-versicolor Iris-versicolor \n", "78 3.0 Iris-versicolor Iris-versicolor \n", "79 2.9 Iris-versicolor Iris-versicolor \n", "80 2.6 Iris-versicolor Iris-versicolor \n", "81 2.4 Iris-versicolor Iris-versicolor \n", "82 2.4 Iris-versicolor Iris-versicolor \n", "83 2.7 Iris-versicolor Iris-versicolor \n", "84 2.7 Iris-versicolor Iris-virginica \n", "85 3.0 Iris-versicolor Iris-versicolor \n", "86 3.4 Iris-versicolor Iris-versicolor \n", "87 3.1 Iris-versicolor Iris-versicolor \n", "88 2.3 Iris-versicolor Iris-versicolor \n", "89 3.0 Iris-versicolor Iris-versicolor \n", "90 2.5 Iris-versicolor Iris-versicolor \n", "91 2.6 Iris-versicolor Iris-versicolor \n", "92 3.0 Iris-versicolor Iris-versicolor \n", "93 2.6 Iris-versicolor Iris-versicolor \n", "94 2.3 Iris-versicolor Iris-versicolor \n", "95 2.7 Iris-versicolor Iris-versicolor \n", "96 3.0 Iris-versicolor Iris-versicolor \n", "97 2.9 Iris-versicolor Iris-versicolor \n", "98 2.9 Iris-versicolor Iris-versicolor \n", "99 2.5 Iris-versicolor Iris-versicolor \n", "100 2.8 Iris-versicolor Iris-versicolor \n", "Rows: 1-100 | Columns: 7" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.predict(iris, name = \"prediction\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can also add the probabilities." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
123
Id
Int
123
PetalLengthCm
Numeric(6,3)
123
PetalWidthCm
Numeric(6,3)
123
SepalLengthCm
Numeric(6,3)
123
SepalWidthCm
Numeric(6,3)
Abc
Species
Varchar(30)
Abc
prediction
Varchar(128)
Abc
prob_irissetosa
Varchar(128)
Abc
prob_irisversicolor
Varchar(128)
Abc
prob_irisvirginica
Varchar(128)
111.40.25.13.5Iris-setosaIris-setosa100
221.40.24.93.0Iris-setosaIris-setosa0.950.0250.025
331.30.24.73.2Iris-setosaIris-setosa100
441.50.24.63.1Iris-setosaIris-setosa100
551.40.25.03.6Iris-setosaIris-setosa100
661.70.45.43.9Iris-setosaIris-setosa100
771.40.34.63.4Iris-setosaIris-setosa100
881.50.25.03.4Iris-setosaIris-setosa100
991.40.24.42.9Iris-setosaIris-setosa100
10101.50.14.93.1Iris-setosaIris-setosa0.950.0250.025
11111.50.25.43.7Iris-setosaIris-setosa100
12121.60.24.83.4Iris-setosaIris-setosa100
13131.40.14.83.0Iris-setosaIris-setosa100
14141.10.14.33.0Iris-setosaIris-setosa100
15151.20.25.84.0Iris-setosaIris-setosa0.9166670.06666670.0166667
16161.50.45.74.4Iris-setosaIris-setosa100
17171.30.45.43.9Iris-setosaIris-setosa100
18181.40.35.13.5Iris-setosaIris-setosa100
19191.70.35.73.8Iris-setosaIris-setosa0.80.20
20201.50.35.13.8Iris-setosaIris-setosa100
21211.70.25.43.4Iris-setosaIris-setosa100
22221.50.45.13.7Iris-setosaIris-setosa100
23231.00.24.63.6Iris-setosaIris-setosa100
24241.70.55.13.3Iris-setosaIris-setosa100
25251.90.24.83.4Iris-setosaIris-setosa0.9250.050.025
26261.60.25.03.0Iris-setosaIris-setosa100
27271.60.45.03.4Iris-setosaIris-setosa100
28281.50.25.23.5Iris-setosaIris-setosa100
29291.40.25.23.4Iris-setosaIris-setosa100
30301.60.24.73.2Iris-setosaIris-setosa100
31311.60.24.83.1Iris-setosaIris-setosa100
32321.50.45.43.4Iris-setosaIris-setosa100
33331.50.15.24.1Iris-setosaIris-setosa100
34341.40.25.54.2Iris-setosaIris-setosa100
35351.50.14.93.1Iris-setosaIris-setosa0.950.0250.025
36361.20.25.03.2Iris-setosaIris-setosa100
37371.30.25.53.5Iris-setosaIris-setosa100
38381.50.14.93.1Iris-setosaIris-setosa0.950.0250.025
39391.30.24.43.0Iris-setosaIris-setosa100
40401.50.25.13.4Iris-setosaIris-setosa100
41411.30.35.03.5Iris-setosaIris-setosa100
42421.30.34.52.3Iris-setosaIris-setosa100
43431.30.24.43.2Iris-setosaIris-setosa100
44441.60.65.03.5Iris-setosaIris-setosa100
45451.90.45.13.8Iris-setosaIris-setosa0.90.10
46461.40.34.83.0Iris-setosaIris-setosa100
47471.60.25.13.8Iris-setosaIris-setosa100
48481.40.24.63.2Iris-setosaIris-setosa100
49491.50.25.33.7Iris-setosaIris-setosa100
50501.40.25.03.3Iris-setosaIris-setosa100
51514.71.47.03.2Iris-versicolorIris-versicolor0.006250.78750.20625
52524.51.56.43.2Iris-versicolorIris-versicolor0.03098290.8309830.138034
53534.91.56.93.1Iris-versicolorIris-versicolor0.08373020.633730.28254
54544.01.35.52.3Iris-versicolorIris-versicolor010
55554.61.56.52.8Iris-versicolorIris-versicolor0.03098290.8309830.138034
56564.51.35.72.8Iris-versicolorIris-versicolor010
57574.71.66.33.3Iris-versicolorIris-versicolor0.04635990.727610.22603
58583.31.04.92.4Iris-versicolorIris-versicolor0.150.5250.325
59594.61.36.62.9Iris-versicolorIris-versicolor0.01944440.9194440.0611111
60603.91.45.22.7Iris-versicolorIris-versicolor0.10.90
61613.51.05.02.0Iris-versicolorIris-versicolor0.10.90
62624.21.55.93.0Iris-versicolorIris-versicolor0.03611110.9277780.0361111
63634.01.06.02.2Iris-versicolorIris-versicolor0.05611110.8877780.0561111
64644.71.46.12.9Iris-versicolorIris-versicolor0.09056620.7034830.205951
65653.61.35.62.9Iris-versicolorIris-versicolor010
66664.41.46.73.1Iris-versicolorIris-versicolor0.03944440.8394440.121111
67674.51.55.63.0Iris-versicolorIris-versicolor010
68684.11.05.82.7Iris-versicolorIris-versicolor0.06666670.8666670.0666667
69694.51.56.22.2Iris-versicolorIris-versicolor0.04264960.8493160.108034
70703.91.15.62.5Iris-versicolorIris-versicolor010
71714.81.85.93.2Iris-versicolorIris-versicolor0.08402780.7819440.134028
72724.01.36.12.8Iris-versicolorIris-versicolor0.04264960.8493160.108034
73734.91.56.32.5Iris-versicolorIris-virginica0.1127290.2627290.624542
74744.71.26.12.8Iris-versicolorIris-versicolor0.09056620.7034830.205951
75754.31.36.42.9Iris-versicolorIris-versicolor0.03098290.8309830.138034
76764.41.46.63.0Iris-versicolorIris-versicolor0.01944440.9194440.0611111
77774.81.46.82.8Iris-versicolorIris-versicolor0.04569440.8269440.127361
78785.01.76.73.0Iris-versicolorIris-versicolor0.103730.553730.34254
79794.51.56.02.9Iris-versicolorIris-versicolor0.05611110.8877780.0561111
80803.51.05.72.6Iris-versicolorIris-versicolor010
81813.81.15.52.4Iris-versicolorIris-versicolor010
82823.71.05.52.4Iris-versicolorIris-versicolor010
83833.91.25.82.7Iris-versicolorIris-versicolor0.06666670.8666670.0666667
84845.11.66.02.7Iris-versicolorIris-virginica0.04861110.4402780.511111
85854.51.55.43.0Iris-versicolorIris-versicolor0.10.80.1
86864.51.66.03.4Iris-versicolorIris-versicolor0.05611110.8877780.0561111
87874.71.56.73.1Iris-versicolorIris-versicolor0.04569440.8269440.127361
88884.41.36.32.3Iris-versicolorIris-versicolor0.04010990.740110.21978
89894.11.35.63.0Iris-versicolorIris-versicolor010
90904.01.35.52.5Iris-versicolorIris-versicolor010
91914.41.25.52.6Iris-versicolorIris-versicolor010
92924.61.46.13.0Iris-versicolorIris-versicolor0.04264960.8493160.108034
93934.01.25.82.6Iris-versicolorIris-versicolor0.06666670.8666670.0666667
94943.31.05.02.3Iris-versicolorIris-versicolor0.10.90
95954.21.35.62.7Iris-versicolorIris-versicolor010
96964.21.25.73.0Iris-versicolorIris-versicolor010
97974.21.35.72.9Iris-versicolorIris-versicolor010
98984.31.36.22.9Iris-versicolorIris-versicolor0.04264960.8493160.108034
99993.01.15.12.5Iris-versicolorIris-versicolor0.10.90
1001004.11.35.72.8Iris-versicolorIris-versicolor010
Rows: 1-100 | Columns: 10
" ], "text/plain": [ " Id PetalLengthCm PetalWidthCm SepalLengthCm \\\\\n", "1 1 1.4 0.2 5.1 \\\\\n", "2 2 1.4 0.2 4.9 \\\\\n", "3 3 1.3 0.2 4.7 \\\\\n", "4 4 1.5 0.2 4.6 \\\\\n", "5 5 1.4 0.2 5.0 \\\\\n", "6 6 1.7 0.4 5.4 \\\\\n", "7 7 1.4 0.3 4.6 \\\\\n", "8 8 1.5 0.2 5.0 \\\\\n", "9 9 1.4 0.2 4.4 \\\\\n", "10 10 1.5 0.1 4.9 \\\\\n", "11 11 1.5 0.2 5.4 \\\\\n", "12 12 1.6 0.2 4.8 \\\\\n", "13 13 1.4 0.1 4.8 \\\\\n", "14 14 1.1 0.1 4.3 \\\\\n", "15 15 1.2 0.2 5.8 \\\\\n", "16 16 1.5 0.4 5.7 \\\\\n", "17 17 1.3 0.4 5.4 \\\\\n", "18 18 1.4 0.3 5.1 \\\\\n", "19 19 1.7 0.3 5.7 \\\\\n", "20 20 1.5 0.3 5.1 \\\\\n", "21 21 1.7 0.2 5.4 \\\\\n", "22 22 1.5 0.4 5.1 \\\\\n", "23 23 1.0 0.2 4.6 \\\\\n", "24 24 1.7 0.5 5.1 \\\\\n", "25 25 1.9 0.2 4.8 \\\\\n", "26 26 1.6 0.2 5.0 \\\\\n", "27 27 1.6 0.4 5.0 \\\\\n", "28 28 1.5 0.2 5.2 \\\\\n", "29 29 1.4 0.2 5.2 \\\\\n", "30 30 1.6 0.2 4.7 \\\\\n", "31 31 1.6 0.2 4.8 \\\\\n", "32 32 1.5 0.4 5.4 \\\\\n", "33 33 1.5 0.1 5.2 \\\\\n", "34 34 1.4 0.2 5.5 \\\\\n", "35 35 1.5 0.1 4.9 \\\\\n", "36 36 1.2 0.2 5.0 \\\\\n", "37 37 1.3 0.2 5.5 \\\\\n", "38 38 1.5 0.1 4.9 \\\\\n", "39 39 1.3 0.2 4.4 \\\\\n", "40 40 1.5 0.2 5.1 \\\\\n", "41 41 1.3 0.3 5.0 \\\\\n", "42 42 1.3 0.3 4.5 \\\\\n", "43 43 1.3 0.2 4.4 \\\\\n", "44 44 1.6 0.6 5.0 \\\\\n", "45 45 1.9 0.4 5.1 \\\\\n", "46 46 1.4 0.3 4.8 \\\\\n", "47 47 1.6 0.2 5.1 \\\\\n", "48 48 1.4 0.2 4.6 \\\\\n", "49 49 1.5 0.2 5.3 \\\\\n", "50 50 1.4 0.2 5.0 \\\\\n", "51 51 4.7 1.4 7.0 \\\\\n", "52 52 4.5 1.5 6.4 \\\\\n", "53 53 4.9 1.5 6.9 \\\\\n", "54 54 4.0 1.3 5.5 \\\\\n", "55 55 4.6 1.5 6.5 \\\\\n", "56 56 4.5 1.3 5.7 \\\\\n", "57 57 4.7 1.6 6.3 \\\\\n", "58 58 3.3 1.0 4.9 \\\\\n", "59 59 4.6 1.3 6.6 \\\\\n", "60 60 3.9 1.4 5.2 \\\\\n", "61 61 3.5 1.0 5.0 \\\\\n", "62 62 4.2 1.5 5.9 \\\\\n", "63 63 4.0 1.0 6.0 \\\\\n", "64 64 4.7 1.4 6.1 \\\\\n", "65 65 3.6 1.3 5.6 \\\\\n", "66 66 4.4 1.4 6.7 \\\\\n", "67 67 4.5 1.5 5.6 \\\\\n", "68 68 4.1 1.0 5.8 \\\\\n", "69 69 4.5 1.5 6.2 \\\\\n", "70 70 3.9 1.1 5.6 \\\\\n", "71 71 4.8 1.8 5.9 \\\\\n", "72 72 4.0 1.3 6.1 \\\\\n", "73 73 4.9 1.5 6.3 \\\\\n", "74 74 4.7 1.2 6.1 \\\\\n", "75 75 4.3 1.3 6.4 \\\\\n", "76 76 4.4 1.4 6.6 \\\\\n", "77 77 4.8 1.4 6.8 \\\\\n", "78 78 5.0 1.7 6.7 \\\\\n", "79 79 4.5 1.5 6.0 \\\\\n", "80 80 3.5 1.0 5.7 \\\\\n", "81 81 3.8 1.1 5.5 \\\\\n", "82 82 3.7 1.0 5.5 \\\\\n", "83 83 3.9 1.2 5.8 \\\\\n", "84 84 5.1 1.6 6.0 \\\\\n", "85 85 4.5 1.5 5.4 \\\\\n", "86 86 4.5 1.6 6.0 \\\\\n", "87 87 4.7 1.5 6.7 \\\\\n", "88 88 4.4 1.3 6.3 \\\\\n", "89 89 4.1 1.3 5.6 \\\\\n", "90 90 4.0 1.3 5.5 \\\\\n", "91 91 4.4 1.2 5.5 \\\\\n", "92 92 4.6 1.4 6.1 \\\\\n", "93 93 4.0 1.2 5.8 \\\\\n", "94 94 3.3 1.0 5.0 \\\\\n", "95 95 4.2 1.3 5.6 \\\\\n", "96 96 4.2 1.2 5.7 \\\\\n", "97 97 4.2 1.3 5.7 \\\\\n", "98 98 4.3 1.3 6.2 \\\\\n", "99 99 3.0 1.1 5.1 \\\\\n", "100 100 4.1 1.3 5.7 \\\\\n", " SepalWidthCm Species prediction prob_irissetosa \\\\\n", "1 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "2 3.0 Iris-setosa Iris-setosa 0.95 \\\\\n", "3 3.2 Iris-setosa Iris-setosa 1 \\\\\n", "4 3.1 Iris-setosa Iris-setosa 1 \\\\\n", "5 3.6 Iris-setosa Iris-setosa 1 \\\\\n", "6 3.9 Iris-setosa Iris-setosa 1 \\\\\n", "7 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "8 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "9 2.9 Iris-setosa Iris-setosa 1 \\\\\n", "10 3.1 Iris-setosa Iris-setosa 0.95 \\\\\n", "11 3.7 Iris-setosa Iris-setosa 1 \\\\\n", "12 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "13 3.0 Iris-setosa Iris-setosa 1 \\\\\n", "14 3.0 Iris-setosa Iris-setosa 1 \\\\\n", "15 4.0 Iris-setosa Iris-setosa 0.916667 \\\\\n", "16 4.4 Iris-setosa Iris-setosa 1 \\\\\n", "17 3.9 Iris-setosa Iris-setosa 1 \\\\\n", "18 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "19 3.8 Iris-setosa Iris-setosa 0.8 \\\\\n", "20 3.8 Iris-setosa Iris-setosa 1 \\\\\n", "21 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "22 3.7 Iris-setosa Iris-setosa 1 \\\\\n", "23 3.6 Iris-setosa Iris-setosa 1 \\\\\n", "24 3.3 Iris-setosa Iris-setosa 1 \\\\\n", "25 3.4 Iris-setosa Iris-setosa 0.925 \\\\\n", "26 3.0 Iris-setosa Iris-setosa 1 \\\\\n", "27 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "28 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "29 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "30 3.2 Iris-setosa Iris-setosa 1 \\\\\n", "31 3.1 Iris-setosa Iris-setosa 1 \\\\\n", "32 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "33 4.1 Iris-setosa Iris-setosa 1 \\\\\n", "34 4.2 Iris-setosa Iris-setosa 1 \\\\\n", "35 3.1 Iris-setosa Iris-setosa 0.95 \\\\\n", "36 3.2 Iris-setosa Iris-setosa 1 \\\\\n", "37 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "38 3.1 Iris-setosa Iris-setosa 0.95 \\\\\n", "39 3.0 Iris-setosa Iris-setosa 1 \\\\\n", "40 3.4 Iris-setosa Iris-setosa 1 \\\\\n", "41 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "42 2.3 Iris-setosa Iris-setosa 1 \\\\\n", "43 3.2 Iris-setosa Iris-setosa 1 \\\\\n", "44 3.5 Iris-setosa Iris-setosa 1 \\\\\n", "45 3.8 Iris-setosa Iris-setosa 0.9 \\\\\n", "46 3.0 Iris-setosa Iris-setosa 1 \\\\\n", "47 3.8 Iris-setosa Iris-setosa 1 \\\\\n", "48 3.2 Iris-setosa Iris-setosa 1 \\\\\n", "49 3.7 Iris-setosa Iris-setosa 1 \\\\\n", "50 3.3 Iris-setosa Iris-setosa 1 \\\\\n", "51 3.2 Iris-versicolor Iris-versicolor 0.00625 \\\\\n", "52 3.2 Iris-versicolor Iris-versicolor 0.0309829 \\\\\n", "53 3.1 Iris-versicolor Iris-versicolor 0.0837302 \\\\\n", "54 2.3 Iris-versicolor Iris-versicolor 0 \\\\\n", "55 2.8 Iris-versicolor Iris-versicolor 0.0309829 \\\\\n", "56 2.8 Iris-versicolor Iris-versicolor 0 \\\\\n", "57 3.3 Iris-versicolor Iris-versicolor 0.0463599 \\\\\n", "58 2.4 Iris-versicolor Iris-versicolor 0.15 \\\\\n", "59 2.9 Iris-versicolor Iris-versicolor 0.0194444 \\\\\n", "60 2.7 Iris-versicolor Iris-versicolor 0.1 \\\\\n", "61 2.0 Iris-versicolor Iris-versicolor 0.1 \\\\\n", "62 3.0 Iris-versicolor Iris-versicolor 0.0361111 \\\\\n", "63 2.2 Iris-versicolor Iris-versicolor 0.0561111 \\\\\n", "64 2.9 Iris-versicolor Iris-versicolor 0.0905662 \\\\\n", "65 2.9 Iris-versicolor Iris-versicolor 0 \\\\\n", "66 3.1 Iris-versicolor Iris-versicolor 0.0394444 \\\\\n", "67 3.0 Iris-versicolor Iris-versicolor 0 \\\\\n", "68 2.7 Iris-versicolor Iris-versicolor 0.0666667 \\\\\n", "69 2.2 Iris-versicolor Iris-versicolor 0.0426496 \\\\\n", "70 2.5 Iris-versicolor Iris-versicolor 0 \\\\\n", "71 3.2 Iris-versicolor Iris-versicolor 0.0840278 \\\\\n", "72 2.8 Iris-versicolor Iris-versicolor 0.0426496 \\\\\n", "73 2.5 Iris-versicolor Iris-virginica 0.112729 \\\\\n", "74 2.8 Iris-versicolor Iris-versicolor 0.0905662 \\\\\n", "75 2.9 Iris-versicolor Iris-versicolor 0.0309829 \\\\\n", "76 3.0 Iris-versicolor Iris-versicolor 0.0194444 \\\\\n", "77 2.8 Iris-versicolor Iris-versicolor 0.0456944 \\\\\n", "78 3.0 Iris-versicolor Iris-versicolor 0.10373 \\\\\n", "79 2.9 Iris-versicolor Iris-versicolor 0.0561111 \\\\\n", "80 2.6 Iris-versicolor Iris-versicolor 0 \\\\\n", "81 2.4 Iris-versicolor Iris-versicolor 0 \\\\\n", "82 2.4 Iris-versicolor Iris-versicolor 0 \\\\\n", "83 2.7 Iris-versicolor Iris-versicolor 0.0666667 \\\\\n", "84 2.7 Iris-versicolor Iris-virginica 0.0486111 \\\\\n", "85 3.0 Iris-versicolor Iris-versicolor 0.1 \\\\\n", "86 3.4 Iris-versicolor Iris-versicolor 0.0561111 \\\\\n", "87 3.1 Iris-versicolor Iris-versicolor 0.0456944 \\\\\n", "88 2.3 Iris-versicolor Iris-versicolor 0.0401099 \\\\\n", "89 3.0 Iris-versicolor Iris-versicolor 0 \\\\\n", "90 2.5 Iris-versicolor Iris-versicolor 0 \\\\\n", "91 2.6 Iris-versicolor Iris-versicolor 0 \\\\\n", "92 3.0 Iris-versicolor Iris-versicolor 0.0426496 \\\\\n", "93 2.6 Iris-versicolor Iris-versicolor 0.0666667 \\\\\n", "94 2.3 Iris-versicolor Iris-versicolor 0.1 \\\\\n", "95 2.7 Iris-versicolor Iris-versicolor 0 \\\\\n", "96 3.0 Iris-versicolor Iris-versicolor 0 \\\\\n", "97 2.9 Iris-versicolor Iris-versicolor 0 \\\\\n", "98 2.9 Iris-versicolor Iris-versicolor 0.0426496 \\\\\n", "99 2.5 Iris-versicolor Iris-versicolor 0.1 \\\\\n", "100 2.8 Iris-versicolor Iris-versicolor 0 \\\\\n", " prob_irisversicolor prob_irisvirginica \n", "1 0 0 \n", "2 0.025 0.025 \n", "3 0 0 \n", "4 0 0 \n", "5 0 0 \n", "6 0 0 \n", "7 0 0 \n", "8 0 0 \n", "9 0 0 \n", "10 0.025 0.025 \n", "11 0 0 \n", "12 0 0 \n", "13 0 0 \n", "14 0 0 \n", "15 0.0666667 0.0166667 \n", "16 0 0 \n", "17 0 0 \n", "18 0 0 \n", "19 0.2 0 \n", "20 0 0 \n", "21 0 0 \n", "22 0 0 \n", "23 0 0 \n", "24 0 0 \n", "25 0.05 0.025 \n", "26 0 0 \n", "27 0 0 \n", "28 0 0 \n", "29 0 0 \n", "30 0 0 \n", "31 0 0 \n", "32 0 0 \n", "33 0 0 \n", "34 0 0 \n", "35 0.025 0.025 \n", "36 0 0 \n", "37 0 0 \n", "38 0.025 0.025 \n", "39 0 0 \n", "40 0 0 \n", "41 0 0 \n", "42 0 0 \n", "43 0 0 \n", "44 0 0 \n", "45 0.1 0 \n", "46 0 0 \n", "47 0 0 \n", "48 0 0 \n", "49 0 0 \n", "50 0 0 \n", "51 0.7875 0.20625 \n", "52 0.830983 0.138034 \n", "53 0.63373 0.28254 \n", "54 1 0 \n", "55 0.830983 0.138034 \n", "56 1 0 \n", "57 0.72761 0.22603 \n", "58 0.525 0.325 \n", "59 0.919444 0.0611111 \n", "60 0.9 0 \n", "61 0.9 0 \n", "62 0.927778 0.0361111 \n", "63 0.887778 0.0561111 \n", "64 0.703483 0.205951 \n", "65 1 0 \n", "66 0.839444 0.121111 \n", "67 1 0 \n", "68 0.866667 0.0666667 \n", "69 0.849316 0.108034 \n", "70 1 0 \n", "71 0.781944 0.134028 \n", "72 0.849316 0.108034 \n", "73 0.262729 0.624542 \n", "74 0.703483 0.205951 \n", "75 0.830983 0.138034 \n", "76 0.919444 0.0611111 \n", "77 0.826944 0.127361 \n", "78 0.55373 0.34254 \n", "79 0.887778 0.0561111 \n", "80 1 0 \n", "81 1 0 \n", "82 1 0 \n", "83 0.866667 0.0666667 \n", "84 0.440278 0.511111 \n", "85 0.8 0.1 \n", "86 0.887778 0.0561111 \n", "87 0.826944 0.127361 \n", "88 0.74011 0.21978 \n", "89 1 0 \n", "90 1 0 \n", "91 1 0 \n", "92 0.849316 0.108034 \n", "93 0.866667 0.0666667 \n", "94 0.9 0 \n", "95 1 0 \n", "96 1 0 \n", "97 1 0 \n", "98 0.849316 0.108034 \n", "99 0.9 0 \n", "100 1 0 \n", "Rows: 1-100 | Columns: 10" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.predict_proba(iris, name = \"prob\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our example forgoes splitting the data into training and testing, which is important for real-world work. Our main goal in this lesson is to look at the metrics used to evaluate classifications. The most famous metric is accuracy: generally speaking, the closer accuracy is to 1, the better the model is. However, taking metrics at face value can lead to incorrect interpretations.\n", "\n", "For example, let's say our goal is to identify bank fraud. Fraudulent activity is relatively rare, so let's say that they represent less than 1% of the data. If we were to predict that there are no frauds in the dataset, we'd end up with an accuracy of 99%. This is why ROC AUC and PRC AUC are more robust metrics.\n", "\n", "That said, a good model is simply a model that might solve a the given problem. In that regard, any model is better than a random one." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.8" } }, "nbformat": 4, "nbformat_minor": 2 }