FAVROPARSER (Parser)

Parses data from an Avro file. The input file must use binary serialization encoding. Use this parser to load data into columnar, flex, and hybrid tables.

For materialized columns in a columnar table, the parser supports all scalar types, one-dimensional arrays of scalar types, and complex types loaded as VMap columns. To load complex types as VMap columns, specify a column type of LONG VARBINARY. To preserve the indexing in complex types, set flatten_maps to false.

The parser favroparser does not support Avro files with separate schema files. The Avro file must have its related schema in the file you are loading.

Syntax

FAVROPARSER ( [parameter='value'[,...]] )

Parameters

flatten_arrays

Boolean, specifies whether to flatten all Avro arrays. Key names are concatenated with nested levels.

This parameter applies only to flex tables or VMap columns. It does not apply to 1D arrays of primitive types that are declared as such.

Default: false

flatten_maps

Boolean, specifies whether to flatten all Avro maps. Key names are concatenated with nested levels.

This parameter applies only to flex tables or VMap columns.

Default: true

flatten_records

Boolean, specifies whether to flatten all Avro records. Key names are concatenated with nested levels.

Default: true

reject_on_materialized_type_error

Boolean, specifies whether to reject any row value for a materialized column that the parser cannot coerce into a compatible data type. See Using Flex Table Parsers.

Default: false

Examples

This example shows how to create and load a flex table with Avro data using favroparser. After loading the data, you can query virtual columns. 

=> CREATE FLEX TABLE avro_basic();
CREATE TABLE
				
=> COPY avro_basic FROM '/home/dbadmin/data/weather.avro' PARSER FAVROPARSER();
Rows Loaded
-------------
5
(1 row)

=> SELECT station, temp, time FROM avro_basic;
station | temp |     time
---------+------+---------------
mohali  | 0    | -619524000000
lucknow | 22   | -619506000000
norwich | -11  | -619484400000
ams     | 111  | -655531200000
baddi   | 78   | -655509600000
(5 rows)

For more information, see Avro Data.