Tips for storing Percona Backup in the cloud

Percona has made a great contribution to the world of open source software. One of the most popular Percona products is backup software MySQL and MongoDB… Such software plays a vital role in backup and development disaster recovery plans

The important point is where the backups are stored. Here, not only the ability to restore from a backup is important, but also the security of storage so that unauthorized persons do not have access to them.

Typically, backups are stored in multiple locations, including the clouds. Cloud storage benefits from cost efficiency, security, and high availability.

Now let’s talk a little more about storing backups in the cloud.

Safety first

The security of storing backups does not mean that encryption is sufficient, or that encryption is required. Security is also implemented by limiting and controlling access to backups, database servers, and storage networks. Please note that encrypting and decrypting files can take a significant amount of time, especially with large amounts of data. The encryption / decryption time also depends on the algorithm used. The more complex the algorithm, the more time and resources are required.

Database encryption

When storing a backup locally, you have several options to improve security and reliability. First of all, this is encryption. For example, creating a logical backup using mysqldump looks like this.

Create file with encryption key

The encryption key file will be used as your pass to encrypt and decrypt the created backup using Percona Backup (Percona XtraBackup or Percona Backup for MongoDB).

$ openssl rand -base64 24 > /root/isolated_directory/keyfile.key

Now that the key has been generated, it needs to be stored in a safe place. You can restrict access to this file and, of course, save it in an isolated and safe place.

$ chmod -R 400 /root/isolated_directory

Create a logical backup

/usr/bin/mysqldump --defaults-file=/etc/my.cnf  --flush-privileges --hex-blob --opt --master-data=2 --single-transaction --skip-lock-tables --triggers --routines --events --gtid  --databases db1 db2 db3 db4  |gzip -6 -c  |  openssl enc  -aes-256-cbc -pass file:/root/isolated_directory/keyfile.key  > /root/mysqldump_2020-12-27_schemaanddata.sql.gz

If you want to know more about database encryption, then read the article Database Backup Encryption – Best Practices

Keeping keys in storage

For increased security, the encryption key / token is best kept in a separate vault… For example, Hashicorp Vault… If you would like to know how to get started with Hashicorp Vault, you can get started from here

If there is a need to store keys with third-party providers, then a good choice would be AWS Secrets Manager… It is a fully managed service from Amazon that integrates with Amazon KMS. You can store and retrieve encryption keys, database passwords and ssh credentials.

Encryption of transmitted data

Placing backups in multiple locations provides redundancy and increases the reliability of your disaster recovery plan. One of the main reasons for storing backups in the cloud is to increase redundancy to reduce the risk of data loss due to data corruption. Clouds are known to be safe, fast, reliable, and cost effective. Cloud storage typically uses fully managed services such as Amazon S3, Google Cloud Storage, and Azure Blob Storage. There are also alternatives: Exoscale, Backblaze and a number of other S3-compatible services. Since files are transferred over open communication channels, make sure to use TLS / SSL. In AWS S3, the bucket can be configured to process requests only over TLS / SSL. For example, as follows:

  "Id": "ExamplePolicy",
  "Version": "2012-10-17",
  "Statement": [
      "Sid": "AllowSSLRequestsOnly",
      "Action": "s3:*",
      "Effect": "Deny",
       "Resource": [
       "Condition": {
         "Bool": {
           "aws:SecureTransport": "false"
       "Principal": "*"

For other cloud storage, for example, Exoscale, you can use the config file .s3cfgsuch as this one:

host_base = sos-{zone}
host_bucket = %(bucket)s.sos-{zone}
access_key = $EXO_SOS_KEY
secret_key = $EXO_SOS_SECRET
use_https = True

Please note that TLS / SSL provides data encryption as it travels to the cloud. TLS is a more secure version of SSL, so use TLS when possible.

Database storage availability

Backups stored in the cloud should be available when needed. If you are using rotation when storing backups in the cloud, make sure they have a sufficient retention period for RPO (Recovery Point Objective) defined by your organization or company. When storing in the cloud, you also have to think about retention periods in terms of resource usage. In this case, it is useful to define the life cycle of the backups. To do this, you can use the tools offered cloud providers

Storage availability means you have the speed and bandwidth you need when you need backups to restore your data. Please note that this must be part of your disaster recovery plan… Be sure to make sure your RTO (Recovery Time Objective) is tested and meets the expected recovery time. Data recovery time, business continuity, and downtime depend on when your database is fully functional, or at least online to serve your most critical queries. The business must run continuously and be continuous.

Economic efficiency

Redundancy and lifecycle management also come with costs. For small and midsize businesses this may not be a problem, but for organizations and companies that manage large amounts of data, it can be a strong case for storing backups in the cloud. But data storage isn’t free. You should identify and select the appropriate file storage plan and provider that suits your requirements.

Some small organizations may store backups in a file storage location such as Google Drive or Microsoft OneDrive.

But do not forget that backups also require metadata and there may also be incremental copies, which are much more convenient to store in object storage. Object storage is also more flexible and provides more features such as ACLs (Access Control Lists), geo-referencing, and flexible cost depending on the storage and resource chosen. Basically, the cost of object storage is very beneficial: you can limit the availability of files, use lifecycle, geo / georeference, tune server performance or throughput when sending and / or receiving data.

Let’s compare the costs with examples.

To get started, let’s take a look at the Amazon AWS S3 Standard storage class and provide a rough cost estimate. Note that S3 Standard is well suited for storing any general purpose data and is generally used for data that is frequently accessed. You can see other storage classes on the page AWS S3

For US N. Virginia, the cost will be as follows:

First 50 TB / month

$ 0.023 per GB

Next 450 TB / month

$ 0.022 per GB

More than 500 TB / month

$ 0.021 per GB

For the US West (Northern California), US West (Northern California), the cost is different.

First 50 TB / month

$ 0.026 per GB

Next 450 TB / month

$ 0.025 per GB

More than 500 TB / month

$ 0.024 per GB

Now you can see that the cost depends on the region.

On the other hand, the price in Google Cloud Storage is for Standard Storage in South Carolina (us-east1) (South Carolina) starts at $ 0.20 per month. But if you move the data from US-EAST1 to NORTHAMERICA-NORTHEAST1, then the price will already be $ 0.01 per GB.

And if you look at Oracle Storage Cloud pricing, then Object Storage starts at $ 0.0255, and Storage Requests starts at $ 0.0034. What’s more, they have Block Volume Performance Units that start at $ 0.0017 and you can choose the type of VPU you want.

So there is no better or worse option. The choice depends on the type of storage, its capabilities, resources, availability, service level agreements (SLAs), and other features that make it more reliable.

Backup automation

When automating database backups, be it Percona Server for MySQL / MariaDB or Percona Server for MongoDB, it’s best to always use software you can rely on.

There are open source scripts you can try using. Or write your own and use Ansible, Chef, Puppet or SaltStack. For corporate use there is ClusterControlthat can manage everything you need. It supports Percona Server for MySQL, Percona Server for MongoDB, MySQL, MariaDB and MongoDB backups. These are open source databases that you can use with Percona Backup Tools supported by ClusterControl

In ClusterControl, you can create a backup policy and run it manually or automatically. The great thing is that it allows you to save backups to multiple locations at once: locally and to the clouds. Currently there is support for AWS, GCP and Azure. You can read the post about how to create backups and then send them to the cloud

Alternatively you can use Backup ninja Is a SaaS (Software-as-a-Service) platform that provides a simple yet effective solution for managing database and file backups. Backup Ninja supports many cloud providers that you may not have used before, and this opens up new possibilities for you. You can use your current cloud storage, which does not have backup automation. Take a look list of cloud storage partners… This service is very well thought out and offers scheduled database backups, database recovery, redundant storage, compression, security, and encryption. Both logical and binary backups are supported. Both full and incremental. There is a file backup. The price starts at $ 40. Backup Ninja is free to use, allowing you to have one backup per agent per day and one restore per agent per day, but only to local storage.

Translation of the article was prepared on the eve of the start of the course “Database”

And we invite everyone to a free demo lesson on the topic: “MySQL NDB Cluster – Sharding”During the lesson, the participants, together with an expert, will analyze the NDB cluster, architecture features and components, and will also work at the stand.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *