What’s New in Vertica 8.1.1: Backup to Amazon S3

As more businesses move their infrastructure to cloud-based and hybrid environments, Vertica has kept pace. In version 8.1.1, Vertica introduced support for backup and restore to and from Amazon’s Simple Storage Service, commonly known as S3. Backup to S3 works the same as any other Vertica backup. As always, backups are incremental. That is, Vertica updates your backup with changes that have occurred since the previous backup. Backup to S3 doesn’t waste resources by copying data that hasn’t changed.

S3 Configuration

The biggest change from S3 support comes in your configuration file. Vertica 8.1.1 introduces the S3 section. This section replaces the Mapping section. The S3 section identifies the S3 bucket that you’re using to store your data, as well as a number of configuration options. [S3] ; This section replaces the [Mapping] section and is required to back up to S3 ; !!Mandatory!! S3 bucket name(no default). s3_backup_path = s3://backup_bucket/database_backup_path/ ; !!Mandatory!! directory used to manage locking during a backup (no default). If the directory is mounted on the initiator host, you should use "[]" instead of the local host name. The file system must support POSIX fcntl flock. s3_backup_file_system_path = []:/home/dbadmin/backup_locks_dir/ ; s3_backup_file_system_path = otherhost.example:/home/dbadmin/backup_locks_dir/ ; Specifies SSL encrypted transfer. ; s3_encrypt_transport = True ; Specifies the number of threads for upload/download - backup ; s3_concurrency_backup = 10 ; Specifies the number of threads for upload/download - restore ; s3_concurrency_restore = 10 The most critical components of the S3 section are these two:

• S3_backup_path
• S3_backup_file_system_path

S3_backup_path

The s3_backup_path identifies the bucket and backup path for your backup to S3. When you perform a backup to S3, all of your nodes use the same S3 bucket.

S3_backup_file_system_path

Those of you familiar with Vertica’s backup and restore capability remember that vbr requires POSIX file locking – a capability not present on S3. To allow for file locking, vbr uses the s3_backup_file_system path. This path enables vbr to create a locked identity file your local or EC2 instance and a duplicate on your S3 backup location. If Vertica is unable to access the identity file, perhaps due to changed permissions, Vertica displays an error and cancels the backup. If the files do not match, perhaps because you changed backup locations, Vertica displays an error and cancels the backup. You can reset your identity file with the command s3-force-init parameter.

A typical S3 locking file reset command takes the following form:

/opt/vertica/bin/vbr –task init –s3-force-init -c filename.ini

Creating the backup

When backing up to S3, you use the same commands and syntax that you use for all Vertica backups. As with any backup, you must initialize your S3 backup location using the init task before creating your first backup. The backup command takes the following form:

vbr –task backup –config-file filename

Here’s an example of a successful backup to S3. [dbadmin@vertica01 ~]$ vbr --task backup –-config-file s3bak.ini Starting backup of database VMart. Participating nodes: v_vmart_node0001, v_vmart_node0002, v_vmart_node0003. Snapshotting database. Snapshot complete. Approximate bytes to copy: 880781754 of 880781754 total. [==================================================] 100% Copying backup metadata. Finalizing backup. Backup complete!

For More Information

For more information on backing up to S3, refer to the Vertica documentation.
For more information on Amazon S3, refer to Amazon’s documentation.