Built-In Pools
Vertica is preconfigured with built-in pools for various system tasks:
Built-in pools can be customized to suit your usage requirements. See ALTER RESOURCE POOL for details on resource pool settings.
GENERAL
A special, catch-all pool used to answer requests that have no specific resource pool associated with them. Any memory left over after memory has been allocated to all other pools is automatically allocated to the GENERAL pool. The MEMORYSIZE parameter of the GENERAL pool is undefined (variable), however, the GENERAL pool must be at least 1GB in size and cannot be smaller than 25% of the memory in the system.
The MAXMEMORYSIZE parameter of the GENERAL pool has special meaning; when set as a % value it represents the percent of total physical RAM on the machine that the Resource Manager can use for queries. By default, it is set to 95%. MAXMEMORYSIZE governs the total amount of RAM that the Resource Manager can use for queries, regardless of whether it is set to a percent or to a specific value (for example, '10GB')
User-defined pools can borrow memory from the GENERAL pool to satisfy requests that need extra memory until the MAXMEMORYSIZE parameter of that pool is reached. If the pool is configured to have MEMORYSIZE equal to MAXMEMORYSIZE, it cannot borrow any memory from the GENERAL pool. When multiple pools request memory from the GENERAL pool, they are granted access to general pool memory according to their priority setting. In this manner, the GENERAL pool provides some elasticity to account for point-in-time deviations from normal usage of individual resource pools.
Vertica recommends reducing the GENERAL pool MAXMEMORYSIZE
if your catalog uses over 5% of overall memory.
BLOBDATA
The BLOBDATA pool controls resource usage for in-memory blobs. In-memory blobs are objects used by a number of the machine learning SQL functions. You should adjust this pool if you plan on processing large machine learning workloads. For information about tuning the pool, see Tuning for Machine Learning.
If a query using the BLOBDATA pool exceeds its query planning budget, then it spills to disk. For more information about tuning your query budget, see Query Budgeting.
DBD
The DBD pool controls resource usage for Database Designer processing. Use of this pool is enabled by configuration parameter DBDUseOnlyDesignerResourcePool, by default set to false.
By default, QUEUETIMEOUT is set to 0 for this pool. When resources are under pressure, this setting causes the DBD to time out immediately, and not be queued to run later. Database Designer then requests the user to run the designer later, when resources are more available.
Do not change QUEUETIMEOUT or any DBD resource pool parameters.
JVM
The JVM pool controls Java Virtual Machine resources used by Java User Defined Extensions. When a Java UDx starts the JVM, it draws resources from the those specified in the JVM resource pool. Vertica does not reserve memory in advance for the JVM pool. When needed, the pool can expand to 10% of physical memory or 2 GB of memory, whichever is smaller. If you are buffering large amounts of data, you may need to increase the size of the JVM resource pool.
You can adjust the size of your JVM resource pool by changing its configuration settings. Unlike other resource pools, the JVM resource pool does not release resources until a session is closed.
METADATA
The pool that tracks memory allocated for catalog data and storage data structures. This pool increases in size as Vertica metadata consumes additional resources. Memory assigned to the METADATA pool is subtracted from the GENERAL pool, enabling the Vertica resource manager to make more effective use of available resources. If the METADATA resource pool reaches 75% of the GENERAL pool, Vertica stops updating METADATA memory size and displays a warning message in the the vertica.log file. You can enable or disable the METADATA pool with the EnableMetadataMemoryTracking general parameter.
If you have created a "dummy" or "swap" resource pool to protect resources for use by your operating system, you can replace that pool with the METADATA pool.
Users cannot change the parameters of the METADATA resource pool.
RECOVERY
The pool used by queries issued when recovering another node of the database. The MAXCONCURRENCY parameter is used to determine how many concurrent recovery threads to use. You can use the PLANNEDCONCURRENCY parameter (by default, set to twice the MAXCONCURRENCY
) to tune how to apportion memory to recovery queries.
See Tuning for Recovery.
REFRESH
The pool used by queries issued by PROJECTION_REFRESHES
operations. Refresh does not currently use multiple concurrent threads; thus, changes to the MAXCONCURRENCY values have no effect.
See Scenario: Tuning for Refresh.
SYSDATA
The pool reserved for temporary storage of intermediate results of queries against system monitoring and catalog tables. If the SYSDATA pool size is too low, Vertica cannot execute queries for large system tables or during high concurrent access to system tables.
MAXMEMORYSIZE of the SYSDATA pool cannot be changed if any of its memory is in use.
SYSQUERY
The pool that runs queries against system monitoring and catalog tables. The SYSQUERY pool reserves resources for system table queries so that they are never blocked by contention for available resources.
TM
The Tuple Mover (TM) pool. You can set the MAXCONCURRENCY parameter for the TM pool to allow more than one concurrent TM operation to occur.
See Tuning Tuple Mover Pool Settings.
WOSDATA
The Write Optimized Store (WOS) resource pool. Data loads to the WOS automatically spill to the ROS when a certain amount of WOS usage is exceeded. Vertica uses the PLANNEDCONCURRENCY parameter of the WOSDATA pool to determine this spill threshold. For instance, if PLANNEDCONCURRENCY of the WOSDATA pool is set to 4, when a load occupies one quarter of the WOS, it spills to the ROS.
The WOSDATA pool is limited to a size of 2GB or 25% of the GENERAL pool's limits, whichever is less. However, when the GENERAL pool has a limit of 20GB or higher, the WOSDATA pool will instead have 2GB of dedicated memory.
You cannot change the MAXMEMORYSIZE parameter of the WOSDATA pool if any of its memory is in use. For example, you cannot change MAXMEMORYSIZE unless you first disable trickle loading jobs and wait until the WOS is empty.
Eon Mode does not use the WOS. When creating or upgrading an Eon Mode database, Vertica sets the default value for MAXMEMORYSIZE and MEMORYSIZE for the WOSDATA resource pool to 0.
See Scenario: Tuning for Continuous Load and Query.
For More Information
For a description of each resource pool parameter, see CREATE RESOURCE POOL.