What’s New in Vertica 8.1.1: Backup to Amazon S3
S3 ConfigurationThe biggest change from S3 support comes in your configuration file. Vertica 8.1.1 introduces the S3 section. This section replaces the Mapping section. The S3 section identifies the S3 bucket that you’re using to store your data, as well as a number of configuration options.
[S3] ; This section replaces the [Mapping] section and is required to back up to S3 ; !!Mandatory!! S3 bucket name(no default). s3_backup_path = s3://backup_bucket/database_backup_path/ ; !!Mandatory!! directory used to manage locking during a backup (no default). If the directory is mounted on the initiator host, you should use "" instead of the local host name. The file system must support POSIX fcntl flock. s3_backup_file_system_path = :/home/dbadmin/backup_locks_dir/ ; s3_backup_file_system_path = otherhost.example:/home/dbadmin/backup_locks_dir/ ; Specifies SSL encrypted transfer. ; s3_encrypt_transport = True ; Specifies the number of threads for upload/download - backup ; s3_concurrency_backup = 10 ; Specifies the number of threads for upload/download - restore ; s3_concurrency_restore = 10The most critical components of the S3 section are these two:
S3_backup_pathThe s3_backup_path identifies the bucket and backup path for your backup to S3. When you perform a backup to S3, all of your nodes use the same S3 bucket.
S3_backup_file_system_pathThose of you familiar with Vertica’s backup and restore capability remember that vbr requires POSIX file locking – a capability not present on S3. To allow for file locking, vbr uses the s3_backup_file_system path. This path enables vbr to create a locked identity file your local or EC2 instance and a duplicate on your S3 backup location. If Vertica is unable to access the identity file, perhaps due to changed permissions, Vertica displays an error and cancels the backup. If the files do not match, perhaps because you changed backup locations, Vertica displays an error and cancels the backup. You can reset your identity file with the command s3-force-init parameter.
A typical S3 locking file reset command takes the following form:
/opt/vertica/bin/vbr –task init –s3-force-init -c filename.ini
Creating the backupWhen backing up to S3, you use the same commands and syntax that you use for all Vertica backups. As with any backup, you must initialize your S3 backup location using the init task before creating your first backup. The backup command takes the following form:
vbr –task backup –config-file filename
Here’s an example of a successful backup to S3.
[dbadmin@vertica01 ~]$ vbr --task backup –-config-file s3bak.ini Starting backup of database VMart. Participating nodes: v_vmart_node0001, v_vmart_node0002, v_vmart_node0003. Snapshotting database. Snapshot complete. Approximate bytes to copy: 880781754 of 880781754 total. [==================================================] 100% Copying backup metadata. Finalizing backup. Backup complete!
For More InformationFor more information on backing up to S3, refer to the Vertica documentation.
For more information on Amazon S3, refer to Amazon’s documentation.