Creating Backups on Amazon S3

Vertica supports the creation of backups on Amazon S3 Standard cloud storage. You can create these backups from your local cluster or from Amazon EC2 virtual servers.

Note: Vertica supports backup and restore from S3. Copycluster and replication to S3 targets is not supported.

Creating an S3 Configuration File

To backup to Amazon S3, you must add an [S3] section to your backup configuration file. For more information, refer to [S3]. Vertica also provides a sample S3 configuration file that you can copy and edit.

Configuring Amazon S3 Storage for Backup

Vertica supports using Amazon S3 cloud storage as a backup location. As with all Vertica backups, Vertica creates incremental backups, meaning each subsequent backup contains only changes that have occurred since you created the first backup. As with any storage location, you must initialize an S3 storage location with the vbr task init.

Because S3 storage does not support file locking, Vertica uses either your local file system or an Amazon EC2 file system to handle file locks during a backup. You identify this location using the s3_backup_file_system_path parameter in your vbr configuration file. During a backup, Vertica creates a locked identity file on your local or EC2 instance, and a duplicate file in your S3 backup location. As long at the files match, Vertica proceeds with the backup, releasing the lock once the backup is complete. As long as the files remain identical, you can use the S3 location for backup and restore tasks.

If the files in your locking location become out of sync with the files in your backup location, backup and restore tasks fail with an error message. You can resolve locking inconsistencies by rerunning the init task with the --s3-force-init parameter.

A typical S3 locking file reset command takes the following form:

/opt/vertica/bin/vbr --task init --s3-force-init -c filename.ini 

Note: If a backup fails, confirm that your Vertica cluster has permission to access your S3 storage location.

Configuring EC2 Authentication for Amazon S3

If you are backing to S3 from an EC2-based cluster, you must provide authentication to your S3 host. Regardless of the authentication type you choose, your credentials do not leave your EC2 cluster. Vertica supports the following authentication types:

AWS credential file - You can manually create a configuration file on your EC2 initiator host at ~/.aws/credentials.

[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY

For more information on credential files, refer to Amazon's Web Services documentation.

IAM role - Create an AWS IAM role and grant that role permission to access your EC2 cluster and S3 resources. For more information, refer to Amazon's Web Services documentation.

Environmental variables - (Recommended) Amazon Web Services provides the following environmental variables:

Use these variables on your initiator to provide authentication to your S3 host. Once your session ends, AWS deletes these variables. For more information, refer to the AWS documentation.