Backup to and Restore from Cloud

In this article, we explore how to configure and manage backups in the cloud using ClusterControl. Storing and retrieving your database backups offsite helps ensure robust disaster recovery strategies, reduces on-premise storage needs, and simplifies the process of restoring data in the event of a system failure or data corruption.

Use cases

Storing backups in the cloud is useful for various scenarios, including:

• Disaster Recovery: Keeping copies of your data offsite in cloud storage helps ensure you have reliable restore points in the event of on-premise system failures.
• Storage Optimization: Offloading backup files to cloud storage can help reduce local disk usage and associated costs.
• Compliance Requirements: Many organizations must maintain offsite backups to meet compliance mandates.
• Scalability: Cloud storage services can easily scale as your backup size grows, eliminating the complexity of managing additional on-premise infrastructure.
• Testing & Development: Quickly restoring backups from cloud storage to dev/test environments makes it easy to replicate real-world data for application testing.

Configuring Cloud Storage

Before you can perform backups to or restores from the cloud, you need to configure storage settings in ClusterControl. Depending on your preferred cloud provider (Amazon S3, Google Cloud Storage, Azure Blob Storage), the general steps are similar.

1. Obtain Your Cloud Credentials

   • For Amazon S3, gather the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION.
   • For Google Cloud, you should have a service account key (JSON file) that you can point to.
   • For Azure, you need to create an storage account and get information such as the application_id, client_secret, tenant_id, etc.
   • For S3-Compatible Storage Provider, required information is similar to the Amazon S3 one.

2. Configure Cloud storage credentials in ClusterControl

In the ClusterControl web interface, navigate to Settings > Cloud storage credentials, and add a new storage profile for your cloud provider. Here you can specify:

• Provider: For example, S3
• Credential/Keyfile: GCP JSON key, or AWS key references
• Region: if needed

Once saved, ClusterControl will be aware of your cloud storage location for backup.

Creating a Backup in the Cloud

Once the cloud storage is configured, perform a backup by following these steps:

1. Navigate to Backup > Create backup. (Or select your cluster and go to Backups)
2. Choose Backup on Demand
3. Select your database cluster
4. Choose the backup method (mysqldump, pg_basebackup, xtrabackup)
5. Enable Upload backup to cloud
6. Follow the next steps
7. Click Create

Monitor the backup process from the Backup tab. Once completed, your backup will be available in the configured cloud storage provider.

Restoring from the Cloud

To restore a backup from the cloud:

1. Navigate to Backup > Backup Storage and verify the stored backups.
2. Select the backup you want to restore.
3. Click Restore and choose the target database cluster.
4. Confirm the restore process.

ClusterControl will download the backup from the cloud and restore it to the specified database cluster.

Automating Backups

To automate backups, set up a schedule backup:

1. Navigate to Backup > Create backup
2. Choose Schedule a Backup
3. Select your database cluster
4. Choose the backup method (mysqldump, pg_basebackup, xtrabackup)
5. Enable Upload backup to cloud
6. Follow the next steps
7. Set the schedule (daily, weekly, etc)
8. Click Create

ClusterControl will automatically create backups at the specified intervals and store them in the cloud.

Verifying Backups

It is a best practice to periodically verify that your backups can be restored successfully. To automate this, use the Verify backup feature in ClusterControl, or perform the following steps manually:

1. Spinning up a temporary replica or development environment
2. Download the backup from the cloud (it can be done from the ClusterControl UI)
3. Restoring the cloud backup
4. Performing integrity checks on the restored data

Accessing Cloud Backups via CLI

Below is an example of how to create and restore backups from the command line. These commands should be run on the ClusterControl server.

• List clusters to find the --cluster-id

  s9s cluster --list --long
  

• Create a backup job to the configured cloud storage

  s9s backup \
      --create \
      --cluster-id=10 \
      --backup-method=xtrabackupfull \
      --cloud-only \
      --cloud-provider="aws" \
      --s3-bucket="my-aws-bucket-eu-west-1" \
      --credential-id=2 \
      --wait
  

• List Available Backups

  s9s backup --list --long
  

• Restore a Backup

  s9s backup \
      --restore \
      --backup-id=5 \
      --cluster-id=10 \
      --wait