Configuring Data Retention Policies

Data Collector retention policies hold the following information:

  • Which component to monitor
  • How much memory to retain
  • How much disk space to retain

As a superuser, you can call meta-function SET_DATA_COLLECTOR_POLICY to modify policies such as change the amount of data to retain. SET_DATA_COLLECTOR_POLICY sets the retention policy for the specified component on all nodes, and lets you specify memory and disk size to retain in kilobytes. Failed nodes receive the new setting when they rejoin the cluster.

For example, the following statement changes memory and disk setting for ResourceAcquisitions from its current setting of 1,000 KB memory and 10,000 KB disk space to 1500 KB and 25000 KB, respectively:

=> SELECT set_data_collector_policy('ResourceAcquisitions', '1500', '25000');
 set_data_collector_policy
---------------------------
 SET
(1 row)

Viewing Retention Policy Settings

Before you change a retention policy, you can view the current setting by calling GET_DATA_COLLECTOR_POLICY. You can also use GET_DATA_COLLECTOR_POLICY to verify changed settings. For example, the following command returns the retention policy for the ResourceAcquisitions component:

=> => SELECT get_data_collector_policy('ResourceAcquisitions');
                          get_data_collector_policy
-----------------------------------------------------------------------------
 1500KB kept in memory, 25000KB kept on disk. Time based retention disabled.
(1 row)

You can query system table DATA_COLLECTOR for the names of Data Collector components. For example, the following query returns all current Data Collector components and a description of each: 

=> SELECT DISTINCT component, description FROM data_collector ORDER BY 1 ASC;

Time-Based Retention

SET_DATA_COLLECTOR_POLICY can set a size restraint only; it can optionally include an interval argument to set disk capacity for either size or time. If you specify an interval, Vertica enforces the setting that is exceeded first, either size or time. Before you include a time restraint, verify that the disk size capacity is sufficiently large.

You can use SET_DATA_COLLECTOR_TIME_POLICY to specify a time restraint only, or to turn off a time restraint that was set earlier with this function.

For example, the following statement sets the RequestsIssued component to 1500 KB memory and 11000 KB on disk, and includes a 3-minute time restraint:

=> SELECT set_data_collector_policy('RequestsIssued', '1500', '11000', '3 minutes'::interval);
 set_data_collector_policy
---------------------------
 SET
(1 row)

The following statement disables the 3-minute retention policy for the RequestsIssued component, using SET_DATA_COLLECTOR_TIME_POLICY:

=> SELECT set_data_collector_time_policy('RequestsIssued', '-1');
 set_data_collector_time_policy
---------------------------
 SET
(1 row)