verticapy.read_csv¶

verticapy.read_csv(path: str, schema: str | None = None, table_name: str | None = None, sep: str | None = None, header: bool = True, header_names: list | None = None, dtype: dict | None = None, na_rep: str | None = None, quotechar: str = '"', escape: str = '\x17', record_terminator: str | None = '\n', trim: bool = True, omit_empty_keys: bool = False, reject_on_duplicate: bool = False, reject_on_empty_key: bool = False, reject_on_materialized_type_error: bool = False, parse_nrows: int = -1, insert: bool = False, temporary_table: bool = False, temporary_local_table: bool = True, gen_tmp_table_name: bool = True, ingest_local: bool = True, genSQL: bool = False, materialize: bool = True) → vDataFrame¶

Ingests a CSV file using flex tables.

Parameters¶

path: str: Absolute path where the CSV file is located.
schema: str, optional: Schema where the CSV file will be ingested.
table_name: str, optional: The final relation/table name. If unspecified, the name is set to the name of the file or parent directory.
sep: str, optional: Column separator.
header: bool, optional: If set to False, the parameter header_names is used to name the different columns.
header_names: list, optional: list of the column names.
dtype: dict, optional: dictionary of the user types. Providing a dictionary can increase ingestion speed and precision; instead of parsing the file to guess the different types, VerticaPy will use the input types.
na_rep: str, optional: Missing values representation.
quotechar: str, optional: Char that encloses the str values.
escape: str, optional: Separator between each record.
record_terminator: str, optional: A single-character value used to specify the end of a record.
trim: bool, optional: boolean, specifies whether to trim white space from header names and key values.
omit_empty_keys: bool, optional: boolean, specifies how the parser handles header keys without values. If True, keys with an empty value in the header row are not loaded.
reject_on_duplicate: bool, optional: boolean, specifies whether to ignore duplicate records (False), or to reject duplicates (True). In either case, the load continues.
reject_on_empty_key: bool, optional: boolean, specifies whether to reject any row containing a key without a value.
reject_on_materialized_type_error: bool, optional: boolean, specifies whether to reject any materialized column value that the parser cannot coerce into a compatible data type.
parse_nrows: int, optional: If this parameter is greater than zero, a new file of parse_nrows rows is created and ingested to identify the data types. It is then dropped and the entire file is ingested. The data types identification will be less precise but this parameter can make the process faster if the file is large.
insert: bool, optional: If set to True, the data is ingested into the specified input relation. Be sure that your file has a header corresponding to the name of the relation columns, otherwise ingestion will fail.
temporary_table: bool, optional: If set to True, a temporary table will be created.
temporary_local_table: bool, optional: If set to True, a temporary local table will be created. The parameter schema must be empty, otherwise this parameter is ignored.
gen_tmp_table_name: bool, optional: Sets the name of the temporary table. This parameter is only used when the parameter temporary_local_table is set to True and if the parameters table_name and schema are unspecified.
ingest_local: bool, optional: If set to True, the file is ingested from the local machine.
genSQL: bool, optional: If set to True, the SQL code for creating the final table is generated but not executed. This is a good way to change the final relation types or to customize the data ingestion.
materialize: bool, optional: If set to True, the flex table is materialized into a table. Otherwise, it will remain a flex table. Flex tables simplify the data ingestion but have worse performace compared to regular tables.

Returns¶

vDataFrame: The vDataFrame of the relation.

Examples¶

In this example, we will first create a CSV file using vDataFrame.to_csv() and ingest it into Vertica database.

We import verticapy:

import verticapy as vp

Hint

By assigning an alias to verticapy, we mitigate the risk of code collisions with other libraries. This precaution is necessary because verticapy uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions from verticapy are used as intended without interfering with functions from other libraries.

We will use the Titanic dataset.

import verticapy.datasets as vpd

data = vpd.load_titanic()

	123 pclass Integer	123 survived Integer	Abc Varchar(164)	Abc sex Varchar(20)	123 age Numeric(8)	123 sibsp Integer	123 parch Integer	Abc ticket Varchar(36)	123 fare Numeric(12)	Abc cabin Varchar(30)	Abc embarked Varchar(20)	Abc boat Varchar(100)	123 body Integer	Abc Varchar(100)
1	1	0		male	71.0	0	0	PC 17609	49.5042	[null]	C	[null]	22
2	1	0		male	45.0	0	0	113784	35.5	T	S	[null]	[null]
3	1	0		male	[null]	0	0	113798	31.0	[null]	S	[null]	[null]
4	1	0		male	17.0	0	0	113059	47.1	[null]	S	[null]	[null]
5	1	0		male	27.0	1	0	13508	136.7792	C89	C	[null]	[null]
6	1	0		male	37.0	1	1	PC 17756	83.1583	E52	C	[null]	[null]
7	1	0		male	31.0	1	0	F.C. 12750	52.0	B71	S	[null]	[null]
8	1	0		male	50.0	1	0	PC 17761	106.425	C86	C	[null]	62
9	1	0		female	36.0	0	0	PC 17531	31.6792	A29	C	[null]	[null]
10	1	0		male	37.0	1	0	113803	53.1	C123	S	[null]	[null]
11	1	0		male	24.0	0	0	PC 17593	79.2	B86	C	[null]	[null]
12	1	0		male	45.0	1	0	36973	83.475	C83	S	[null]	[null]
13	1	0		male	40.0	0	0	112059	0.0	B94	S	[null]	110
14	1	0		male	42.0	0	0	113038	42.5	B11	S	[null]	[null]
15	1	0		male	[null]	0	0	17463	51.8625	E46	S	[null]	[null]
16	1	0		male	42.0	1	0	113789	52.0	[null]	S	[null]	38
17	1	0		male	[null]	0	0	PC 17600	30.6958	[null]	C	14	[null]
18	1	0		male	29.0	0	0	113501	30.0	D6	S	[null]	126
19	1	0		male	46.0	0	0	13050	75.2417	C6	C	[null]	292
20	1	0		male	54.0	0	0	17463	51.8625	E46	S	[null]	175
21	1	0		male	47.0	0	0	113796	42.4	[null]	S	[null]	[null]
22	1	0		male	58.0	0	2	35273	113.275	D48	C	[null]	122
23	1	0		male	45.5	0	0	113043	28.5	C124	S	[null]	166
24	1	0		male	29.0	1	0	113776	66.6	C2	S	[null]	[null]
25	1	0		male	47.0	0	0	110465	52.0	C110	S	[null]	207
26	1	0		male	38.0	0	0	19972	0.0	[null]	S	[null]	[null]
27	1	0		male	22.0	0	0	PC 17760	135.6333	[null]	C	[null]	232
28	1	0		male	31.0	0	0	PC 17590	50.4958	A24	S	[null]	[null]
29	1	0		male	50.0	1	0	13507	55.9	E44	S	[null]	[null]
30	1	0		male	56.0	0	0	17764	30.6958	A7	C	[null]	[null]
31	1	0		male	57.0	1	0	PC 17569	146.5208	B78	C	[null]	[null]
32	1	0		female	63.0	1	0	PC 17483	221.7792	C55 C57	S	[null]	[null]
33	1	0		male	61.0	0	0	36963	32.3208	D50	S	[null]	46
34	1	0		male	21.0	0	1	35281	77.2875	D26	S	[null]	169
35	1	0		male	51.0	0	1	PC 17597	61.3792	[null]	C	[null]	[null]
36	1	1		female	63.0	1	0	13502	77.9583	D7	S	10	[null]
37	1	1		female	32.0	0	0	11813	76.2917	D15	C	8	[null]
38	1	1		female	58.0	0	0	113783	26.55	C103	S	8	[null]
39	1	1		female	44.0	0	0	PC 17610	27.7208	B4	C	6	[null]
40	1	1		female	41.0	0	0	16966	134.5	E40	C	3	[null]
41	1	1		female	53.0	0	0	PC 17606	27.4458	[null]	C	6	[null]
42	1	1		male	36.0	0	1	PC 17755	512.3292	B51 B53 B55	C	3	[null]
43	1	1		female	58.0	0	1	PC 17755	512.3292	B51 B53 B55	C	3	[null]
44	1	1		male	11.0	1	2	113760	120.0	B96 B98	S	4	[null]
45	1	1		female	76.0	1	0	19877	78.85	C46	S	6	[null]
46	1	1		female	[null]	0	1	113505	55.0	E33	S	6	[null]
47	1	1		female	39.0	1	1	PC 17756	83.1583	E49	C	14	[null]
48	1	1		female	27.0	1	2	F.C. 12750	52.0	B71	S	3	[null]
49	1	1		female	[null]	0	0	17421	110.8833	[null]	C	4	[null]
50	1	1		female	35.0	0	0	113503	211.5	C130	C	4	[null]
51	1	1		female	22.0	0	1	112378	59.4	[null]	C	7	[null]
52	1	1		female	25.0	1	0	11765	55.4417	E50	C	5	[null]
53	1	1		male	48.0	1	0	PC 17572	76.7292	D33	C	3	[null]
54	1	1		female	35.0	1	0	36973	83.475	C83	S	D	[null]
55	1	1		male	27.0	0	0	PC 17572	76.7292	D49	C	3	[null]
56	1	1		female	24.0	0	0	11767	83.1583	C54	C	7	[null]
57	1	1		female	52.0	1	1	12749	93.5	B69	S	3	[null]
58	1	1		female	44.0	0	1	111361	57.9792	B18	C	4	[null]
59	1	1		female	15.0	0	1	24160	211.3375	B5	S	2	[null]
60	1	1		male	30.0	1	0	13236	57.75	C78	C	11	[null]
61	1	1		female	31.0	1	0	35273	113.275	D36	C	6	[null]
62	1	1		female	39.0	0	0	PC 17758	108.9	C105	C	8	[null]
63	1	1		female	22.0	0	1	113509	61.9792	B36	C	5	[null]
64	1	1		male	52.0	0	0	113786	30.5	C104	S	6	[null]
65	1	1		female	43.0	0	1	24160	211.3375	B3	S	2	[null]
66	1	1		female	33.0	0	0	110152	86.5	B77	S	8	[null]
67	1	1		male	45.0	1	1	16966	134.5	E34	C	3	[null]
68	1	1		female	40.0	1	1	16966	134.5	E34	C	3	[null]
69	1	1		male	48.0	1	0	19996	52.0	C126	S	5 7	[null]
70	1	1		female	[null]	0	0	PC 17585	79.2	[null]	C	D	[null]
71	1	1		female	35.0	0	0	PC 17755	512.3292	[null]	C	3	[null]
72	1	1		female	60.0	1	0	110813	75.25	D37	C	5	[null]
73	1	1		male	21.0	0	1	PC 17597	61.3792	[null]	C	A	[null]
74	2	0		male	23.0	0	0	C.A. 31030	10.5	[null]	S	[null]	[null]
75	2	0		male	28.0	0	0	244358	26.0	[null]	S	[null]	[null]
76	2	0		male	60.0	1	1	29750	39.0	[null]	S	[null]	[null]
77	2	0		female	44.0	1	0	244252	26.0	[null]	S	[null]	[null]
78	2	0		male	29.0	1	0	2003	26.0	[null]	S	[null]	[null]
79	2	0		male	18.0	0	0	S.O.C. 14879	73.5	[null]	S	[null]	[null]
80	2	0		male	18.0	0	0	S.O.C. 14879	73.5	[null]	S	[null]	[null]
81	2	0		male	54.0	0	0	28403	26.0	[null]	S	[null]	[null]
82	2	0		male	18.0	0	0	236171	13.0	[null]	S	[null]	[null]
83	2	0		male	36.0	0	0	229236	13.0	[null]	S	[null]	236
84	2	0		male	34.0	1	0	28664	21.0	[null]	S	[null]	[null]
85	2	0		male	21.0	1	0	28133	11.5	[null]	S	[null]	[null]
86	2	0		male	21.0	1	0	28134	11.5	[null]	S	[null]	[null]
87	2	0		male	24.0	0	0	233866	13.0	[null]	S	[null]	155
88	2	0		male	34.0	0	0	12233	13.0	[null]	S	[null]	[null]
89	2	0		male	30.0	0	0	250653	13.0	[null]	S	[null]	75
90	2	0		male	44.0	0	0	248746	13.0	[null]	S	[null]	35
91	2	0		male	49.0	1	2	220845	65.0	[null]	S	[null]	[null]
92	2	0		male	21.0	2	0	S.O.C. 14879	73.5	[null]	S	[null]	[null]
93	2	0		male	21.0	0	0	S.O.C. 14879	73.5	[null]	S	[null]	[null]
94	2	0		female	60.0	1	0	24065	26.0	[null]	S	[null]	[null]
95	2	0		male	24.0	2	0	C.A. 31029	31.5	[null]	S	[null]	[null]
96	2	0		male	22.0	2	0	C.A. 31029	31.5	[null]	S	[null]	[null]
97	2	0		male	35.0	0	0	233734	12.35	[null]	Q	[null]	[null]
98	2	0		male	31.0	0	0	C.A. 18723	10.5	[null]	S	[null]	165
99	2	0		male	36.0	0	0	SC/Paris 2163	12.875	D	C	[null]	[null]
100	2	0		male	[null]	0	0	SC/A.3 2861	15.5792	[null]	C	[null]	[null]

Rows: 1-100 | Columns: 14

Note

VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.

Let’s convert the vDataFrame to a CSV.

data[0:20].to_csv(
    path = "titanic_subset.csv",
)

Our CSV file is ready to be ingested in database.

Let’s generate, the SQL needed to create the Table.

from verticapy.core.parsers.csv import read_csv

read_csv(
    path = "titanic_subset.csv",
    table_name = "titanic_subset",
    schema = "public",
    quotechar = '"',
    sep = ",",
    na_rep = "",
    genSQL = True,
)

Out[4]: 
['CREATE TABLE "public"."titanic_subset"("pclass" Integer, "survived" Integer, "name" Varchar(64), "sex" Varchar(20), "age" Numeric(8,4), "sibsp" Integer, "parch" Integer, "ticket" Varchar(20), "fare" Numeric(11,6), "cabin" Varchar(20), "embarked" Varchar(20), "boat" Integer, "body" Integer, "home.dest" Varchar(64))',
 'COPY "public"."titanic_subset"("pclass", "survived", "name", "sex", "age", "sibsp", "parch", "ticket", "fare", "cabin", "embarked", "boat", "body", "home.dest") FROM LOCAL \'titanic_subset.csv\' UNCOMPRESSED DELIMITER \',\' NULL \'\' ENCLOSED BY \'"\' RECORD TERMINATOR E\'\\n\' ESCAPE AS \'\x17\' SKIP 1']

Note

When genSQL flag is set to True, the SQL code for creating the final table is generated but not executed. This is a good way to change the final relation types or to customize the data ingestion.

Now, we will ingest the CSV file into the Vertica database.

read_csv(
    path = "titanic_subset.csv",
    table_name = "titanic_subset",
    schema = "public",
    quotechar = '"',
    sep = ",",
    na_rep = "",
)

	123 pclass Integer	123 survived Integer	Abc name Varchar(64)	Abc sex Varchar(20)	123 age Numeric(10)	123 sibsp Integer	123 parch Integer	Abc ticket Varchar(20)	123 fare Numeric(13)	Abc cabin Varchar(20)	Abc embarked Varchar(20)	123 boat Integer	123 body Integer	Abc home.dest Varchar(64)
1	1	0	Blackwell, Mr. Stephen Weart	male	45.0	0	0	113784	35.5	T	S	[null]	[null]	Trenton, NJ
2	1	0	Clark, Mr. Walter Miller	male	27.0	1	0	13508	136.7792	C89	C	[null]	[null]	Los Angeles, CA
3	1	0	Davidson, Mr. Thornton	male	31.0	1	0	F.C. 12750	52.0	B71	S	[null]	[null]	Montreal, PQ
4	1	0	Head, Mr. Christopher	male	42.0	0	0	113038	42.5	B11	S	[null]	[null]	London / Middlesex
5	1	0	Long, Mr. Milton Clyde	male	29.0	0	0	113501	30.0	D6	S	[null]	126	Springfield, MA
6	1	0	Compton, Mr. Alexander Taylor Jr	male	37.0	1	1	PC 17756	83.1583	E52	C	[null]	[null]	Lakewood, NJ
7	1	0	Evans, Miss. Edith Corse	female	36.0	0	0	PC 17531	31.6792	A29	C	[null]	[null]	New York, NY
8	1	0	Harris, Mr. Henry Birkhardt	male	45.0	1	0	36973	83.475	C83	S	[null]	[null]	New York, NY
9	1	0	McCarthy, Mr. Timothy J	male	54.0	0	0	17463	51.8625	E46	S	[null]	175	Dorchester, MA
10	1	0	Cairns, Mr. Alexander	male	[null]	0	0	113798	31.0	[null]	S	[null]	[null]	[null]
11	1	0	Douglas, Mr. Walter Donald	male	50.0	1	0	PC 17761	106.425	C86	C	[null]	62	Deephaven, MN / Cedar Rapids, IA
12	1	0	Giglio, Mr. Victor	male	24.0	0	0	PC 17593	79.2	B86	C	[null]	[null]	[null]
13	1	0	Hilliard, Mr. Herbert Henry	male	[null]	0	0	17463	51.8625	E46	S	[null]	[null]	Brighton, MA
14	1	0	Artagaveytia, Mr. Ramon	male	71.0	0	0	PC 17609	49.5042	[null]	C	[null]	22	Montevideo, Uruguay
15	1	0	Carrau, Mr. Jose Pedro	male	17.0	0	0	113059	47.1	[null]	S	[null]	[null]	Montevideo, Uruguay
16	1	0	Futrelle, Mr. Jacques Heath	male	37.0	1	0	113803	53.1	C123	S	[null]	[null]	Scituate, MA
17	1	0	Harrison, Mr. William	male	40.0	0	0	112059	0.0	B94	S	[null]	110	[null]
18	1	0	Holverson, Mr. Alexander Oskar	male	42.0	1	0	113789	52.0	[null]	S	[null]	38	New York, NY
19	1	0	Hoyt, Mr. William Fisher	male	[null]	0	0	PC 17600	30.6958	[null]	C	14	[null]	New York, NY
20	1	0	McCaffry, Mr. Thomas Francis	male	46.0	0	0	13050	75.2417	C6	C	[null]	292	Vancouver, BC

Rows: 20 | Columns: 14

Important

A limited number of rows, determined by the parse_nrows parameter, is ingested. If your dataset is large and you want to ingest the entire dataset, increase its value.

Let’s specify data types using dtype parameter.

read_csv(
    path = "titanic_subset.csv",
    table_name = "titanic_sub_dtypes",
    schema = "public",
    quotechar = '"',
    sep = ",",
    na_rep = "",
    dtype = {
        "pclass": "Integer",
        "survived": "Integer",
        "name": "Varchar(164)",
        "sex": "Varchar(20)",
        "age": "Numeric(6,3)",
        "sibsp": "Integer",
        "parch": "Integer",
        "ticket": "Varchar(36)",
        "fare": "Numeric(10,5)",
        "cabin": "Varchar(30)",
        "embarked": "Varchar(20)",
        "boat": "Varchar(100)",
        "body": "Integer",
        "home.dest": "Varchar(100)",
    },
)

	123 pclass Integer	123 survived Integer	Abc name Varchar(164)	Abc sex Varchar(20)	123 age Numeric(8)	123 sibsp Integer	123 parch Integer	Abc ticket Varchar(36)	123 fare Numeric(12)	Abc cabin Varchar(30)	Abc embarked Varchar(20)	Abc boat Varchar(100)	123 body Integer	Abc home.dest Varchar(100)
1	1	0	Artagaveytia, Mr. Ramon	male	71.0	0	0	PC 17609	49.5042	[null]	C	[null]	22	Montevideo, Uruguay
2	1	0	Blackwell, Mr. Stephen Weart	male	45.0	0	0	113784	35.5	T	S	[null]	[null]	Trenton, NJ
3	1	0	Cairns, Mr. Alexander	male	[null]	0	0	113798	31.0	[null]	S	[null]	[null]	[null]
4	1	0	Carrau, Mr. Jose Pedro	male	17.0	0	0	113059	47.1	[null]	S	[null]	[null]	Montevideo, Uruguay
5	1	0	Clark, Mr. Walter Miller	male	27.0	1	0	13508	136.7792	C89	C	[null]	[null]	Los Angeles, CA
6	1	0	Compton, Mr. Alexander Taylor Jr	male	37.0	1	1	PC 17756	83.1583	E52	C	[null]	[null]	Lakewood, NJ
7	1	0	Davidson, Mr. Thornton	male	31.0	1	0	F.C. 12750	52.0	B71	S	[null]	[null]	Montreal, PQ
8	1	0	Douglas, Mr. Walter Donald	male	50.0	1	0	PC 17761	106.425	C86	C	[null]	62	Deephaven, MN / Cedar Rapids, IA
9	1	0	Evans, Miss. Edith Corse	female	36.0	0	0	PC 17531	31.6792	A29	C	[null]	[null]	New York, NY
10	1	0	Futrelle, Mr. Jacques Heath	male	37.0	1	0	113803	53.1	C123	S	[null]	[null]	Scituate, MA
11	1	0	Giglio, Mr. Victor	male	24.0	0	0	PC 17593	79.2	B86	C	[null]	[null]	[null]
12	1	0	Harris, Mr. Henry Birkhardt	male	45.0	1	0	36973	83.475	C83	S	[null]	[null]	New York, NY
13	1	0	Harrison, Mr. William	male	40.0	0	0	112059	0.0	B94	S	[null]	110	[null]
14	1	0	Head, Mr. Christopher	male	42.0	0	0	113038	42.5	B11	S	[null]	[null]	London / Middlesex
15	1	0	Hilliard, Mr. Herbert Henry	male	[null]	0	0	17463	51.8625	E46	S	[null]	[null]	Brighton, MA
16	1	0	Holverson, Mr. Alexander Oskar	male	42.0	1	0	113789	52.0	[null]	S	[null]	38	New York, NY
17	1	0	Hoyt, Mr. William Fisher	male	[null]	0	0	PC 17600	30.6958	[null]	C	14	[null]	New York, NY
18	1	0	Long, Mr. Milton Clyde	male	29.0	0	0	113501	30.0	D6	S	[null]	126	Springfield, MA
19	1	0	McCaffry, Mr. Thomas Francis	male	46.0	0	0	13050	75.2417	C6	C	[null]	292	Vancouver, BC
20	1	0	McCarthy, Mr. Timothy J	male	54.0	0	0	17463	51.8625	E46	S	[null]	175	Dorchester, MA

Rows: 20 | Columns: 14

Note

You can ingest multiple CSV files into the Vertica database by using the following syntax.

read_csv(
    path = "*.csv",
    table_name = "titanic_multi_files",
    schema = "public",
    quotechar = '"',
    sep = ",",
    na_rep = "",
)

Note

The read_csv() function offers various additional parameters and options. Check the documentation to explore its capabilities, such as the ability to automatically guess the separator or ingest a specific number of rows.