Rebalancing Data Across Nodes

Vertica can rebalance your database when you add or remove nodes. As a superuser, you can manually trigger a rebalance with Administration Tools, SQL functions, or the Management Console.

A rebalance operation can take some time, depending on the cluster size, and the number of projections and the amount of data they contain. You should allow the process to complete uninterrupted. If you must cancel the operation, call CANCEL_REBALANCE_CLUSTER.

Why Rebalance?

Rebalancing is useful or even necessary after you perform one of the following operations:

• Change the size of the cluster by adding or removing nodes.
• Mark one or more nodes as ephemeral in preparation of removing them from the cluster.

• Change the scaling factor of an elastic cluster, which determines the number of storage containers used to store a projection across the database.

• Set the control node size or realign control nodes on a large cluster layout.
• Specify more than 120 nodes in your initial Vertica cluster configuration.
• Modify a fault group by adding or removing nodes.

General Rebalancing Tasks

When you rebalance a database cluster, Vertica performs the following tasks for all projections, segmented and unsegmented alike:

• Distributes data based on:

  • User-defined fault groups, if specified
  • Large cluster automatic fault groups
• Ignores node-specific distribution specifications in projection definitions. Node rebalancing always distributes data across all nodes.
• When rebalancing is complete, sets the Ancient History Mark the greatest allowable epoch (now).

Vertica rebalances segmented and unsegmented projections differently, as described below.

Rebalancing Segmented Projections

For each segmented projection, Vertica performs the following tasks:

1. Copies and renames projection buddies and distributes them evenly across all nodes. The renamed projections share the same base name.
2. Refreshes the new projections.
3. Drops the original projections.

Rebalancing Unsegmented Projections

For each unsegmented projection, Vertica performs the following tasks:

If adding nodes:

• Creates projection buddies on them.
• Maps the new projections to their shared name in the database catalog.

If dropping nodes: drops the projection buddies from them.

K-safety and Rebalancing

Until rebalancing completes, Vertica operates with the existing K-safe value. After rebalancing completes, Vertica operates with the K-safe value specified during the rebalance operation. The new K-safe value must be equal to or higher than current K-safety. Vertica does not support downgrading K-safety and returns a warning if you try to reduce it from its current value. For more information, see Lowering K-Safety to Enable Node Removal.

Rebalancing Failure and Projections

If a failure occurs while rebalancing the database, you can rebalance again. If the cause of the failure has been resolved, the rebalance operation continues from where it failed. However, a failed data rebalance can result in projections becoming out of date.

To locate any such projections, query the system table PROJECTIONS as follows:

=> SELECT projection_name, anchor_table_name, is_prejoin, is_up_to_date FROM projections 
   WHERE is_up_to_date = false;

To remove out-of-date projections, use DROP PROJECTION.

Temporary Tables

Node rebalancing has no effect on projections of temporary tables.