Trickle Loading
Trickle loading is the process of loading data into your database periodically. You can use trickle loading to create a more sustainable load process that generates fewer ROS containers.
The Vertica database uses the transaction isolation level of READ COMMITTED. This level allows users to see the most recently committed data without holding any locks. READ COMMITTED also allows new data to be loaded while concurrent queries are running.
To tune the trickle-loading operations:
- Adjust batch size. Generally, larger load batches are more efficient.
- Alter the moveout/mergeout parameters:
- MergeOutInterval
- MoveOutInterval
- MoveOutMaxAgeTime
- MoveOutSizePct
- Limit the load process to a specific resource so that other resources are available to queries. To do so, create a resource pool specifically for the trickle-loading process.