verticapy.read_json¶
- verticapy.read_json(path: str, schema: str | None = None, table_name: str | None = None, usecols: list | None = None, new_name: dict | None = None, insert: bool = False, start_point: str = None, record_terminator: str = None, suppress_nonalphanumeric_key_chars: bool = False, reject_on_materialized_type_error: bool = False, reject_on_duplicate: bool = False, reject_on_empty_key: bool = False, flatten_maps: bool = True, flatten_arrays: bool = False, temporary_table: bool = False, temporary_local_table: bool = True, gen_tmp_table_name: bool = True, ingest_local: bool = True, genSQL: bool = False, materialize: bool = True, use_complex_dt: bool = False, is_avro: bool = False) vDataFrame¶
Ingests a JSON file using flex tables.
Parameters¶
- path: str
Absolute path where the JSON file is located.
- schema: str, optional
Schema where the JSON file will be ingested.
- table_name: str, optional
Final relation name.
- usecols: list, optional
listof the JSON parameters to ingest. The other parameters will be ignored. If empty, all the JSON parameters will be ingested.- new_name: dict, optional
Dictionary of the new column names. If the JSON file is nested, it is recommended to change the final names because special characters will be included in the new column names. For example,
{"param": {"age": 3, "name": Badr}, "date": 1993-03-11}will create 3 columns: “param.age”, “param.name” and “date”. You can rename these columns using thenew_nameparameter with the followingdictionary:{"param.age": "age", "param.name": "name"}- insert: bool, optional
If set to
True, the data is ingested into the input relation. The JSON parameters must be the same as the input relation otherwise they will not be ingested. If set toTrue,table_namecannot be empty.- start_point: str, optional
str, name of a key in the JSON load data at which to begin parsing. The parser ignores all data before thestart_pointvalue. The value is loaded for each object in the file. The parser processes data after the first instance, and up to the second, ignoring any remaining data.- record_terminator: str, optional
When set, any invalid JSON records are skipped and parsing continues with the next record. Records must be terminated uniformly. For example, if your input file has JSON records terminated by newline characters, set this parameter to
\n. If any invalid JSON records exist, parsing continues after the nextrecord_terminator. Even if the data does not contain invalid records, specifying an explicit record terminator can improve load performance by allowing cooperative parse and apportioned load to operate more efficiently. When you omit this parameter, parsing ends at the first invalid JSON record.- suppress_nonalphanumeric_key_chars: bool, optional
boolean, whether to suppress non-alphanumeric characters in JSON key values. The parser replaces these characters with an underscore (_) when this parameter isTrue.- reject_on_materialized_type_error: bool, optional
boolean, whether to reject a data row that contains a materialized column value that cannot be coerced into a compatible data type. If the value isFalseand the type cannot be coerced, the parser sets the value in that column toNone. If the column is a strongly-typed complex type, as opposed to a flexible complex type, then a type mismatch anywhere in the complex type causes the entire column to be treated as a mismatch. The parser does not partially load complex types.- reject_on_duplicate: bool, optional
boolean, whether to ignore duplicate records (False), or to reject duplicates (True). In either case, the load continues.- reject_on_empty_key: bool, optional
boolean, whether to reject any row containing a field key without a value.- flatten_maps: bool, optional
boolean, whether to flatten sub-maps within the JSON data, separating map levels with a period (.). This value affects all data in the load, including nested maps.- flatten_arrays: bool, optional
boolean, whether to convert lists to sub-maps withintegerkeys. When lists are flattened, key names are concatenated in the same way as maps.listsare not flattened by default. This value affects all data in the load, including nestedlists.- temporary_table: bool, optional
If set to
True, a temporary table will be created.- temporary_local_table: bool, optional
If set to
True, a temporary local table will be created. The parameterschemamust be empty, otherwise this parameter is ignored.- gen_tmp_table_name: bool, optional
Sets the name of the temporary table. This parameter is only used when the parameter
temporary_local_tableis set toTrueand if the parameterstable_nameandschemaare unspecified.- ingest_local: bool, optional
If set to
True, the file will be ingested from the local machine.- genSQL: bool, optional
If set to
True, the SQL code for creating the final table is generated but not executed. This is a good way to change the final relation types or to customize the data ingestion.- materialize: bool, optional
If set to
True, the flex table is materialized into a table. Otherwise, it will remain a flex table. Flex tables simplify the data ingestion but have worse performace compared to regular tables.- use_complex_dt: bool, optional
boolean, whether the input data file has complex structure. If set toTrue, most of the other parameters are ignored.
Returns¶
- vDataFrame
The
vDataFrameof the relation.
Examples¶
In this example, we will first create a JSON file using
vDataFrame.to_json()and ingest it into Vertica database.We import
verticapy:import verticapy as vp
Hint
By assigning an alias to
verticapy, we mitigate the risk of code collisions with other libraries. This precaution is necessary because verticapy uses commonly known function names like “average” and “median”, which can potentially lead to naming conflicts. The use of an alias ensures that the functions fromverticapyare used as intended without interfering with functions from other libraries.We will use the Titanic dataset.
import verticapy.datasets as vpd data = vpd.load_titanic()
123pclass123survivedAbcAbcsex123age123sibsp123parchAbcticket123fareAbccabinAbcembarkedAbcboat123bodyAbc1 1 0 male 71.0 0 0 PC 17609 49.5042 [null] C [null] 22 2 1 0 male 45.0 0 0 113784 35.5 T S [null] [null] 3 1 0 male [null] 0 0 113798 31.0 [null] S [null] [null] 4 1 0 male 17.0 0 0 113059 47.1 [null] S [null] [null] 5 1 0 male 27.0 1 0 13508 136.7792 C89 C [null] [null] 6 1 0 male 37.0 1 1 PC 17756 83.1583 E52 C [null] [null] 7 1 0 male 31.0 1 0 F.C. 12750 52.0 B71 S [null] [null] 8 1 0 male 50.0 1 0 PC 17761 106.425 C86 C [null] 62 9 1 0 female 36.0 0 0 PC 17531 31.6792 A29 C [null] [null] 10 1 0 male 37.0 1 0 113803 53.1 C123 S [null] [null] 11 1 0 male 24.0 0 0 PC 17593 79.2 B86 C [null] [null] 12 1 0 male 45.0 1 0 36973 83.475 C83 S [null] [null] 13 1 0 male 40.0 0 0 112059 0.0 B94 S [null] 110 14 1 0 male 42.0 0 0 113038 42.5 B11 S [null] [null] 15 1 0 male [null] 0 0 17463 51.8625 E46 S [null] [null] 16 1 0 male 42.0 1 0 113789 52.0 [null] S [null] 38 17 1 0 male [null] 0 0 PC 17600 30.6958 [null] C 14 [null] 18 1 0 male 29.0 0 0 113501 30.0 D6 S [null] 126 19 1 0 male 46.0 0 0 13050 75.2417 C6 C [null] 292 20 1 0 male 54.0 0 0 17463 51.8625 E46 S [null] 175 21 1 0 male 47.0 0 0 113796 42.4 [null] S [null] [null] 22 1 0 male 58.0 0 2 35273 113.275 D48 C [null] 122 23 1 0 male 45.5 0 0 113043 28.5 C124 S [null] 166 24 1 0 male 29.0 1 0 113776 66.6 C2 S [null] [null] 25 1 0 male 47.0 0 0 110465 52.0 C110 S [null] 207 26 1 0 male 38.0 0 0 19972 0.0 [null] S [null] [null] 27 1 0 male 22.0 0 0 PC 17760 135.6333 [null] C [null] 232 28 1 0 male 31.0 0 0 PC 17590 50.4958 A24 S [null] [null] 29 1 0 male 50.0 1 0 13507 55.9 E44 S [null] [null] 30 1 0 male 56.0 0 0 17764 30.6958 A7 C [null] [null] 31 1 0 male 57.0 1 0 PC 17569 146.5208 B78 C [null] [null] 32 1 0 female 63.0 1 0 PC 17483 221.7792 C55 C57 S [null] [null] 33 1 0 male 61.0 0 0 36963 32.3208 D50 S [null] 46 34 1 0 male 21.0 0 1 35281 77.2875 D26 S [null] 169 35 1 0 male 51.0 0 1 PC 17597 61.3792 [null] C [null] [null] 36 1 1 female 63.0 1 0 13502 77.9583 D7 S 10 [null] 37 1 1 female 32.0 0 0 11813 76.2917 D15 C 8 [null] 38 1 1 female 58.0 0 0 113783 26.55 C103 S 8 [null] 39 1 1 female 44.0 0 0 PC 17610 27.7208 B4 C 6 [null] 40 1 1 female 41.0 0 0 16966 134.5 E40 C 3 [null] 41 1 1 female 53.0 0 0 PC 17606 27.4458 [null] C 6 [null] 42 1 1 male 36.0 0 1 PC 17755 512.3292 B51 B53 B55 C 3 [null] 43 1 1 female 58.0 0 1 PC 17755 512.3292 B51 B53 B55 C 3 [null] 44 1 1 male 11.0 1 2 113760 120.0 B96 B98 S 4 [null] 45 1 1 female 76.0 1 0 19877 78.85 C46 S 6 [null] 46 1 1 female [null] 0 1 113505 55.0 E33 S 6 [null] 47 1 1 female 39.0 1 1 PC 17756 83.1583 E49 C 14 [null] 48 1 1 female 27.0 1 2 F.C. 12750 52.0 B71 S 3 [null] 49 1 1 female [null] 0 0 17421 110.8833 [null] C 4 [null] 50 1 1 female 35.0 0 0 113503 211.5 C130 C 4 [null] 51 1 1 female 22.0 0 1 112378 59.4 [null] C 7 [null] 52 1 1 female 25.0 1 0 11765 55.4417 E50 C 5 [null] 53 1 1 male 48.0 1 0 PC 17572 76.7292 D33 C 3 [null] 54 1 1 female 35.0 1 0 36973 83.475 C83 S D [null] 55 1 1 male 27.0 0 0 PC 17572 76.7292 D49 C 3 [null] 56 1 1 female 24.0 0 0 11767 83.1583 C54 C 7 [null] 57 1 1 female 52.0 1 1 12749 93.5 B69 S 3 [null] 58 1 1 female 44.0 0 1 111361 57.9792 B18 C 4 [null] 59 1 1 female 15.0 0 1 24160 211.3375 B5 S 2 [null] 60 1 1 male 30.0 1 0 13236 57.75 C78 C 11 [null] 61 1 1 female 31.0 1 0 35273 113.275 D36 C 6 [null] 62 1 1 female 39.0 0 0 PC 17758 108.9 C105 C 8 [null] 63 1 1 female 22.0 0 1 113509 61.9792 B36 C 5 [null] 64 1 1 male 52.0 0 0 113786 30.5 C104 S 6 [null] 65 1 1 female 43.0 0 1 24160 211.3375 B3 S 2 [null] 66 1 1 female 33.0 0 0 110152 86.5 B77 S 8 [null] 67 1 1 male 45.0 1 1 16966 134.5 E34 C 3 [null] 68 1 1 female 40.0 1 1 16966 134.5 E34 C 3 [null] 69 1 1 male 48.0 1 0 19996 52.0 C126 S 5 7 [null] 70 1 1 female [null] 0 0 PC 17585 79.2 [null] C D [null] 71 1 1 female 35.0 0 0 PC 17755 512.3292 [null] C 3 [null] 72 1 1 female 60.0 1 0 110813 75.25 D37 C 5 [null] 73 1 1 male 21.0 0 1 PC 17597 61.3792 [null] C A [null] 74 2 0 male 23.0 0 0 C.A. 31030 10.5 [null] S [null] [null] 75 2 0 male 28.0 0 0 244358 26.0 [null] S [null] [null] 76 2 0 male 60.0 1 1 29750 39.0 [null] S [null] [null] 77 2 0 female 44.0 1 0 244252 26.0 [null] S [null] [null] 78 2 0 male 29.0 1 0 2003 26.0 [null] S [null] [null] 79 2 0 male 18.0 0 0 S.O.C. 14879 73.5 [null] S [null] [null] 80 2 0 male 18.0 0 0 S.O.C. 14879 73.5 [null] S [null] [null] 81 2 0 male 54.0 0 0 28403 26.0 [null] S [null] [null] 82 2 0 male 18.0 0 0 236171 13.0 [null] S [null] [null] 83 2 0 male 36.0 0 0 229236 13.0 [null] S [null] 236 84 2 0 male 34.0 1 0 28664 21.0 [null] S [null] [null] 85 2 0 male 21.0 1 0 28133 11.5 [null] S [null] [null] 86 2 0 male 21.0 1 0 28134 11.5 [null] S [null] [null] 87 2 0 male 24.0 0 0 233866 13.0 [null] S [null] 155 88 2 0 male 34.0 0 0 12233 13.0 [null] S [null] [null] 89 2 0 male 30.0 0 0 250653 13.0 [null] S [null] 75 90 2 0 male 44.0 0 0 248746 13.0 [null] S [null] 35 91 2 0 male 49.0 1 2 220845 65.0 [null] S [null] [null] 92 2 0 male 21.0 2 0 S.O.C. 14879 73.5 [null] S [null] [null] 93 2 0 male 21.0 0 0 S.O.C. 14879 73.5 [null] S [null] [null] 94 2 0 female 60.0 1 0 24065 26.0 [null] S [null] [null] 95 2 0 male 24.0 2 0 C.A. 31029 31.5 [null] S [null] [null] 96 2 0 male 22.0 2 0 C.A. 31029 31.5 [null] S [null] [null] 97 2 0 male 35.0 0 0 233734 12.35 [null] Q [null] [null] 98 2 0 male 31.0 0 0 C.A. 18723 10.5 [null] S [null] 165 99 2 0 male 36.0 0 0 SC/Paris 2163 12.875 D C [null] [null] 100 2 0 male [null] 0 0 SC/A.3 2861 15.5792 [null] C [null] [null] Rows: 1-100 | Columns: 14Note
VerticaPy offers a wide range of sample datasets that are ideal for training and testing purposes. You can explore the full list of available datasets in the Datasets, which provides detailed information on each dataset and how to use them effectively. These datasets are invaluable resources for honing your data analysis and machine learning skills within the VerticaPy environment.
Let’s convert the
vDataFrameto a JSON file.data[0:20].to_json( path = "titanic_subset.json", )
Let’s ingest the json file into the Vertica database.
from verticapy.core.parsers.json import read_json read_json( path = "titanic_subset.json", table_name = "titanic_subset", schema = "public", )
123boat123bodyAbccabin123ageAbchome.destAbcembarkedAbcsex123parch123pclass123fareAbcname123sibsp123survivedAbcticket1 [null] [null] E46 [null] Brighton, MA S male 0 1 51.8625 Hilliard, Mr. Herbert Henry 0 0 17463 2 [null] [null] [null] 17.0 Montevideo, Uruguay S male 0 1 47.1 Carrau, Mr. Jose Pedro 0 0 113059 3 [null] 62 C86 50.0 Deephaven, MN / Cedar Rapids, IA C male 0 1 106.425 Douglas, Mr. Walter Donald 1 0 PC 17761 4 [null] [null] B11 42.0 London / Middlesex S male 0 1 42.5 Head, Mr. Christopher 0 0 113038 5 [null] [null] [null] [null] [null] S male 0 1 31.0 Cairns, Mr. Alexander 0 0 113798 6 [null] 22 [null] 71.0 Montevideo, Uruguay C male 0 1 49.5042 Artagaveytia, Mr. Ramon 0 0 PC 17609 7 [null] 292 C6 46.0 Vancouver, BC C male 0 1 75.2417 McCaffry, Mr. Thomas Francis 0 0 13050 8 [null] [null] A29 36.0 New York, NY C female 0 1 31.6792 Evans, Miss. Edith Corse 0 0 PC 17531 9 [null] [null] C83 45.0 New York, NY S male 0 1 83.475 Harris, Mr. Henry Birkhardt 1 0 36973 10 [null] [null] E52 37.0 Lakewood, NJ C male 1 1 83.1583 Compton, Mr. Alexander Taylor Jr 1 0 PC 17756 11 [null] [null] T 45.0 Trenton, NJ S male 0 1 35.5 Blackwell, Mr. Stephen Weart 0 0 113784 12 [null] 110 B94 40.0 [null] S male 0 1 0.0 Harrison, Mr. William 0 0 112059 13 [null] [null] B71 31.0 Montreal, PQ S male 0 1 52.0 Davidson, Mr. Thornton 1 0 F.C. 12750 14 [null] [null] B86 24.0 [null] C male 0 1 79.2 Giglio, Mr. Victor 0 0 PC 17593 15 [null] [null] C123 37.0 Scituate, MA S male 0 1 53.1 Futrelle, Mr. Jacques Heath 1 0 113803 16 [null] [null] C89 27.0 Los Angeles, CA C male 0 1 136.7792 Clark, Mr. Walter Miller 1 0 13508 17 [null] 38 [null] 42.0 New York, NY S male 0 1 52.0 Holverson, Mr. Alexander Oskar 1 0 113789 18 [null] 126 D6 29.0 Springfield, MA S male 0 1 30.0 Long, Mr. Milton Clyde 0 0 113501 19 [null] 175 E46 54.0 Dorchester, MA S male 0 1 51.8625 McCarthy, Mr. Timothy J 0 0 17463 20 14 [null] [null] [null] New York, NY C male 0 1 30.6958 Hoyt, Mr. William Fisher 0 0 PC 17600 Rows: 1-20 | Columns: 14Let’s ingest the json and rename some columns.
read_json( path = "titanic_subset.json", table_name = "titanic_sub_newnames", schema = "public", new_name = { "fields.fare": "fare", "fields.sex": "sex", }, )
123boat123bodyAbccabin123ageAbchome.destAbcembarkedAbcsex123parch123pclass123fareAbcname123sibsp123survivedAbcticket1 [null] [null] E46 [null] Brighton, MA S male 0 1 51.8625 Hilliard, Mr. Herbert Henry 0 0 17463 2 [null] [null] [null] 17.0 Montevideo, Uruguay S male 0 1 47.1 Carrau, Mr. Jose Pedro 0 0 113059 3 [null] 62 C86 50.0 Deephaven, MN / Cedar Rapids, IA C male 0 1 106.425 Douglas, Mr. Walter Donald 1 0 PC 17761 4 [null] [null] B11 42.0 London / Middlesex S male 0 1 42.5 Head, Mr. Christopher 0 0 113038 5 [null] [null] [null] [null] [null] S male 0 1 31.0 Cairns, Mr. Alexander 0 0 113798 6 [null] 22 [null] 71.0 Montevideo, Uruguay C male 0 1 49.5042 Artagaveytia, Mr. Ramon 0 0 PC 17609 7 [null] 292 C6 46.0 Vancouver, BC C male 0 1 75.2417 McCaffry, Mr. Thomas Francis 0 0 13050 8 [null] [null] A29 36.0 New York, NY C female 0 1 31.6792 Evans, Miss. Edith Corse 0 0 PC 17531 9 [null] [null] C83 45.0 New York, NY S male 0 1 83.475 Harris, Mr. Henry Birkhardt 1 0 36973 10 [null] [null] E52 37.0 Lakewood, NJ C male 1 1 83.1583 Compton, Mr. Alexander Taylor Jr 1 0 PC 17756 11 [null] [null] T 45.0 Trenton, NJ S male 0 1 35.5 Blackwell, Mr. Stephen Weart 0 0 113784 12 [null] 110 B94 40.0 [null] S male 0 1 0.0 Harrison, Mr. William 0 0 112059 13 [null] [null] B71 31.0 Montreal, PQ S male 0 1 52.0 Davidson, Mr. Thornton 1 0 F.C. 12750 14 [null] [null] B86 24.0 [null] C male 0 1 79.2 Giglio, Mr. Victor 0 0 PC 17593 15 [null] [null] C123 37.0 Scituate, MA S male 0 1 53.1 Futrelle, Mr. Jacques Heath 1 0 113803 16 [null] [null] C89 27.0 Los Angeles, CA C male 0 1 136.7792 Clark, Mr. Walter Miller 1 0 13508 17 [null] 38 [null] 42.0 New York, NY S male 0 1 52.0 Holverson, Mr. Alexander Oskar 1 0 113789 18 [null] 126 D6 29.0 Springfield, MA S male 0 1 30.0 Long, Mr. Milton Clyde 0 0 113501 19 [null] 175 E46 54.0 Dorchester, MA S male 0 1 51.8625 McCarthy, Mr. Timothy J 0 0 17463 20 14 [null] [null] [null] New York, NY C male 0 1 30.6958 Hoyt, Mr. William Fisher 0 0 PC 17600 Rows: 1-20 | Columns: 14Let’s ingest only two columns from the json.
read_json( path = "titanic_subset.json", table_name = "titanic_sub_usecols", schema = "public", usecols = [ "fields.fare", "fields.sex", ], )
123boat123bodyAbccabin123ageAbchome.destAbcembarkedAbcsex123parch123pclass123fareAbcname123sibsp123survivedAbcticket1 [null] [null] E46 [null] Brighton, MA S male 0 1 51.8625 Hilliard, Mr. Herbert Henry 0 0 17463 2 [null] [null] [null] 17.0 Montevideo, Uruguay S male 0 1 47.1 Carrau, Mr. Jose Pedro 0 0 113059 3 [null] 62 C86 50.0 Deephaven, MN / Cedar Rapids, IA C male 0 1 106.425 Douglas, Mr. Walter Donald 1 0 PC 17761 4 [null] [null] B11 42.0 London / Middlesex S male 0 1 42.5 Head, Mr. Christopher 0 0 113038 5 [null] [null] [null] [null] [null] S male 0 1 31.0 Cairns, Mr. Alexander 0 0 113798 6 [null] 22 [null] 71.0 Montevideo, Uruguay C male 0 1 49.5042 Artagaveytia, Mr. Ramon 0 0 PC 17609 7 [null] 292 C6 46.0 Vancouver, BC C male 0 1 75.2417 McCaffry, Mr. Thomas Francis 0 0 13050 8 [null] [null] A29 36.0 New York, NY C female 0 1 31.6792 Evans, Miss. Edith Corse 0 0 PC 17531 9 [null] [null] C83 45.0 New York, NY S male 0 1 83.475 Harris, Mr. Henry Birkhardt 1 0 36973 10 [null] [null] E52 37.0 Lakewood, NJ C male 1 1 83.1583 Compton, Mr. Alexander Taylor Jr 1 0 PC 17756 11 [null] [null] T 45.0 Trenton, NJ S male 0 1 35.5 Blackwell, Mr. Stephen Weart 0 0 113784 12 [null] 110 B94 40.0 [null] S male 0 1 0.0 Harrison, Mr. William 0 0 112059 13 [null] [null] B71 31.0 Montreal, PQ S male 0 1 52.0 Davidson, Mr. Thornton 1 0 F.C. 12750 14 [null] [null] B86 24.0 [null] C male 0 1 79.2 Giglio, Mr. Victor 0 0 PC 17593 15 [null] [null] C123 37.0 Scituate, MA S male 0 1 53.1 Futrelle, Mr. Jacques Heath 1 0 113803 16 [null] [null] C89 27.0 Los Angeles, CA C male 0 1 136.7792 Clark, Mr. Walter Miller 1 0 13508 17 [null] 38 [null] 42.0 New York, NY S male 0 1 52.0 Holverson, Mr. Alexander Oskar 1 0 113789 18 [null] 126 D6 29.0 Springfield, MA S male 0 1 30.0 Long, Mr. Milton Clyde 0 0 113501 19 [null] 175 E46 54.0 Dorchester, MA S male 0 1 51.8625 McCarthy, Mr. Timothy J 0 0 17463 20 14 [null] [null] [null] New York, NY C male 0 1 30.6958 Hoyt, Mr. William Fisher 0 0 PC 17600 Rows: 1-20 | Columns: 14Note
You can ingest multiple JSON files into the Vertica database by using the following syntax.
read_json( path = "*.json", table_name = "titanic_multi_files", schema = "public", )
See also
read_file(): Ingests an input file into the Vertica DB.read_avro(): Ingests a AVRO file into the Vertica DB.read_csv(): Ingests a CSV file into the Vertica DB.read_pandas(): Ingests thepandas.DataFrameinto the Vertica DB.