Expanding Your Cluster with New Nodes

Posted June 14, 2018 by Soniya Shah, Information Developer

Does your business need more storage? Consider adding nodes to your database if you need more storage. Vertica recommends that you add all new nodes at the same time. You can use this checklist to add nodes. These are basic steps. There are additional options discussed in the Vertica documentation.
Step Task Results
1 Perform a full local backup of your existing database. $ vbr -t backup --config $FULLBAK_CONFIGBy default, no output appears, other than the progress bar. To include additional progress information, use the –debug option, with a value between 1–3.   Specify a config file when you run the backup utility. If no file exists, the backup fails with an error. Resolve the error and proceed to Step 2.
2 Drop older or unused table partitions. => SELECT DROP_PARTITION(table-name, partition value); If the table partition is dropped, proceed to Step 3. If the table partition is not dropped, check the table name and partition value and try again.
3 Verify that local segmentation is disabled. If it is not disabled, disable it. => SELECT DISABLE_LOCAL_SEGMENTS(); After you have confirmed that local segmentation is disabled, proceed to Step 4.
4 Check the network bandwidth and CPU performance of your cluster. $ /opt/vertica/bin/vnetperf $ /opt/vertica/bin/vcpuperf Measure the latency and throughput of your network between nodes and measure the speed of consistency of your hard drives. If the latency and throughput is lower than your initial benchmarks, contact the system administrator to fix the problems causing lower benchmarks.
5 Check if you have enough storage (at least 40% of the size of your database) to perform the rebalance. To get a snapshot of each node, review the following fields in the HOST_RESOURCES system table: => SELECT host_name, disk_space_used_mb, disk_space_total_mb disk_space_free_mb FROM host_resources; Confirm that the amount of disk, column, and projection storage for each node is sufficient to perform a rebalance. If so, proceed to Step 6. If you do not have enough storage, try to reduce your catalog size. For more information, see Rebalance Taking a Long Time checklist.
6 Minimize the DML operations on tables that are part of a rebalance. When a rebalance takes a lock on a table, a load fails. If your rebalance might contend with ETL jobs, increase the value of the configuration parameter, LockTimeout. => ALTER SESSION SET LockTimeout = value After you have addressed ongoing DML operations, proceed to Step 7. The default value is 300 seconds (5 minutes).
7 If your database has more than one node already, you can add a node without stopping the server. However, if you are adding a node to a single-node installation, then you must shut down both the database and spread. $ /opt/vertica/bin/admintools -t stop_db -d db-name After you stop the database, proceed to Step 8.
8 Configure hosts and add hosts to the cluster using the update_vertica script. a. Use the update_vertica script to add hosts. $ /opt/vertica/sbin/update_vertica --add-hosts host(s) --rpm packageb. Add nodes to your database. $ admintools -t db_add_node -d sampleDB -p 'password' –s node(s) The –s option allows you to specify the order of the nodes you are adding. After adding nodes to the database, Vertica automatically distributes updated configuration files to the rest of the nodes in the cluster and starts the process of rebalancing data in the cluster. After the hosts and nodes are added, proceed to Step 9.
9 You can monitor the rebalance progress of individual tables. => SELECT table_name, separated_percent, transferred_percent FROM REBALANCE_TABLE_STATUS; After monitoring, proceed to Step 10.
10 Check if rebalancing completed successfully. => SELECT operation_status FROM REBALANCE_OPERATIONS; If operation_status = COMPLETE, the rebalance completed without errors. If all nodes in the cluster have rebalanced without error, you have completed this checklist. If your rebalance did not complete successfully, contact Vertica Technical Support.

Learn More