Viewing Parquet Export Events More Easily

Posted January 27, 2020 by Jim Knicely, Vertica Principal Solution Architect

Helpful Tips in blue text with magnifying glass

The EXPORT TO PARQUET command exports a table, columns from a table, or query results to files in the Parquet format. When you run EXPORT TO PARQUET information about the files created during the export is stored in the Vertica log. It’s no fun combing through a Vertica log looking for those particular records.

Good news! As of Vertica 9.3.1, the V_MONITOR.UDX_EVENTS system table now also records the events logged during the export!

Example:

dbadmin=> SELECT * FROM test;
 c1
----
  1
  2
  3
(3 rows)

dbadmin=> EXPORT TO PARQUET(DIRECTORY='/home/dbadmin/parq/') AS SELECT * FROM test;
 Rows Exported
---------------
             3
(1 row)

dbadmin=> SELECT created, file, rows FROM v_monitor.udx_events WHERE udx_name = 'ParquetExport';
            created            |                                       file                                       | rows
-------------------------------+----------------------------------------------------------------------------------+------
 2020-01-27 10:05:30.166202-05 | /home/dbadmin/parqhUGvipCI/ad3a94ae-v_test_db_node0002-139833020229376-0.parquet | 3
(1 row)

Helpful Links:
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/SQLReferenceManual/Statements/EXPORTTOPARQUET.htm
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/HadoopIntegrationGuide/NativeFormats/ExportMonitor.htm
https://www.vertica.com/docs/9.3.x/HTML/Content/Authoring/SQLReferenceManual/SystemTables/MONITOR/UDX_EVENTS.htm

Have Fun!