Working with Data Collection Logs

Upon startup, a Vertica database creates a DataCollector directory within the /catalog directory.

The DataCollector directory holds the disk-based data collection logs, where retained data is kept in files named <component>.<timestamp>.log. Vertica might maintain several log files, per component, at any given time. See Querying Data Collector Tables for an example of how to view this information.

Also upon startup, Vertica creates two additional files, per component, in the DataCollector directory. These are SQL files that contain examples on how to load Data Collector data into another Vertica instance. These files are:

  • CREATE_<component>_TABLE.sql — contains the SQL DDL needed to create a table into which Data Collector logs for the component can be loaded.
  • COPY_<component>_TABLE.sql — contains example SQL to load (using COPY commands) the data log files into the table that the CREATE script creates.

Two functions let you manipulate these log files.

If you want to ... See these topics in the SQL Reference Manual

Clear memory and disk records from Data Collector retention and reset collection statistics

CLEAR_DATA_COLLECTOR()

Flush the Data Collector logs

FLUSH_DATA_COLLECTOR()

Retrieve a list of all current Data Collector components

V_MONITOR.DATA_COLLECTOR

Clearing the Data Collector

If you want to clear the Data Collector of all memory and disk records and reset the collection statistics in the V_MONITOR.DATA_COLLECTOR system table, call the CLEAR_DATA_COLLECTOR() function. You can clear records for all components on all nodes or you can specify individual components, one at a time.

To clear records on a single component, pass the function the component argument. For example, the following command clears records for the ResourceAcquisitions component only, returning a result of CLEAR (success):

=> SELECT clear_data_collector('ResourceAcquisitions');
 clear_data_collector
----------------------
 CLEAR
(1 row)

The following command clears the Data Collector records for all components on all nodes:

=> SELECT clear_data_collector();
 clear_data_collector
----------------------
 CLEAR
(1 row)

After you clear the DataCollector log, the information is no longer available for querying.

Flushing Data Collector Logs

If you want to flush Data Collector information for all components or for an individual component, use the FLUSH_DATA_COLLECTOR() function. This function waits until memory logs are moved to disk and flushes the Data Collector, synchronizing the log with the disk storage:

To flush data collection for all components on all nodes:

=> SELECT flush_data_collector();
 flush_data_collector
----------------------
 FLUSH
(1 row)

To flush records on a single component, pass a component argument to the function. For example, the following command flushes the ResourceAcquisitions component:

=> SELECT flush_data_collector('ResourceAcquisitions');
 flush_data_collector
----------------------
 FLUSH
(1 row)

See Also