Backup and RecoveryFeedback
The operator can orchestrate a continuous backup infrastructure
that is based on the Barman tool. Instead
of using the classical architecture with a Barman server, which
backup many PostgreSQL instances, the operator will use the
As a result, base backups will be tarballs. Both base backups and WAL files
can be compressed and encrypted.
For this, it is required an image with
You can use the image
quay.io/enterprisedb/postgresql for this scope,
as it is composed of a community PostgreSQL image and the latest
A backup is performed from a primary or a designated primary instance in a
Cluster (please refer to
for more information about designated primary instances).
Cloud Native PostgreSQL does not currently manage the deletion of backup files from the backup object store. The retention policy feature will be merged from Barman to Barman Cloud in the future. For the time being, it is your responsibility to configure retention policies directly on the object store.
You can archive the backup files in any service that is supported by the Barman Cloud infrastructure. That is:
You can also use any compatible implementation of the supported services.
The required setup depends on the chosen storage provider and is discussed in the following sections.
You will need the following information about your environment:
ACCESS_KEY_ID: the ID of the access key that will be used to upload files in S3
ACCESS_SECRET_KEY: the secret part of the previous access key
ACCESS_SESSION_TOKEN: the optional session token in case it is required
The access key used must have permission to upload files in the bucket. Given that, you must create a k8s secret with the credentials, and you can do that with the following command:
The credentials will be stored inside Kubernetes and will be encrypted if encryption at rest is configured in your installation.
Given that secret, you can configure your cluster like in the following example:
The destination path can be every URL pointing to a folder where
the instance can upload the WAL files, e.g.
In case you're using S3-compatible object storage, like MinIO or Linode Object Storage, you can specify an endpoint instead of using the default S3 one.
In this example, it will use the
bucket bucket of Linode in the region
Suppose you configure an Object Storage provider which uses a certificated signed with a private CA,
like when using OpenShift or MinIO via HTTPS. In that case, you need to set the option
referring to a secret containing the CA bundle so that Barman can verify the certificate correctly.
If you want ConfigMaps and Secrets to be automatically reloaded by instances, you can
add a label with key
k8s.enterprisedb.io/reload to it, otherwise you will have to reload
the instances using the
kubectl cnp reload subcommand.
Optionally, you can use MinIO Gateway as a common interface which relays backup objects to other cloud storage solutions, like S3 or GCS. For more information, please refer to MinIO official documentation.
Specifically, the Cloud Native PostgreSQL cluster can directly point to a local MinIO Gateway as an endpoint, using previously created credentials and service.
MinIO secrets will be used by both the PostgreSQL cluster and the MinIO instance. Therefore you must create them in the same namespace:
Cloud Object Storage credentials will be used only by MinIO Gateway in this case.
In order to allow PostgreSQL to reach MinIO Gateway, it is necessary to create a
ClusterIP service on port
9000 bound to the MinIO Gateway instance.
At the time of writing this documentation, the official MinIO Operator
for Kubernetes does not support the gateway feature. As such, we will use a
The MinIO deployment will use cloud storage credentials to upload objects to the remote bucket and relay backup files to different locations.
Here is an example using AWS S3 as Cloud Object Storage:
Proceed by configuring MinIO Gateway service as the
endpointURL in the
definition, then choose a bucket name to replace
s3://BUCKET_NAME/ the presence of archived WAL files before
proceeding with a backup.
In order to access your storage account, you will need one of the following combinations of credentials:
- Connection String
- Storage account name and Storage account access key
- Storage account name and Storage account SAS Token.
The credentials need to be stored inside a Kubernetes Secret, adding data entries only when needed. The following command performs that:
The credentials will be encrypted at rest, if this feature is enabled in the used Kubernetes cluster.
Given the previous secret, the provided credentials can be injected inside the cluster configuration:
When using the Azure Blob Storage, the
destinationPath fulfills the following
<container>/<blob>. The account name,
which is also called storage account name, is included in the used host name.
If you are using a different implementation of the Azure Blob Storage APIs,
destinationPath will have the following structure:
In that case,
<account-name> is the first component of the path.
This is required if you are testing the Azure support via the Azure Storage Emulator or Azurite.
To request a new backup, you need to create a new Backup resource like the following one:
The operator will start to orchestrate the cluster to take the
required backup using
barman-cloud-backup. You can check
the backup status using the plain
kubectl describe backup <name>
When the backup has been completed, the phase will be
like in the following example:
This feature will not backup the secrets for the superuser and the application user. The secrets are supposed to be backed up as part of the standard backup procedures for the Kubernetes cluster.
You can also schedule your backups periodically by creating a
ScheduledBackup. The latter is similar to a
Backup but with an added field, called
This field is a cron schedule specification, which follows the same format used in Kubernetes CronJobs.
This is an example of a scheduled backup:
The above example will schedule a backup every day at midnight.
Backup frequency might impact your recovery time object (RTO) after a disaster which requires a full or Point-In-Time recovery operation. Our advice is that you regularly test your backups by recovering them, and then measuring the time it takes to recover from scratch so that you can refine your RTO predictability. Recovery time is influenced by the size of the base backup and the amount of WAL files that need to fetched from the archive and replayed during recovery (remember that WAL archiving is what enables continuous backup in PostgreSQL!). Based on our experience, a weekly base backup is more than enough for most cases - while it is extremely rare to schedule backups more frequently than once a day.
ScheduledBackups can be suspended if needed by setting
this will stop any new backup to be scheduled as long as the option is set to false.
In case you want to issue a backup as soon as the ScheduledBackup resource is created
you can set
WAL archiving is enabled as soon as you choose a destination path and you configure your cloud credentials.
If required, you can choose to compress WAL files as soon as they are uploaded and/or encrypt them:
You can configure the encryption directly in your bucket, and the operator will use it unless you override it in the cluster configuration.
You can use the data uploaded to the object storage to bootstrap a
new cluster from a backup. The operator will orchestrate the recovery
process using the
When a backup is completed, the corresponding Kubernetes resource will contain every information needed to restore it, just like in the following example:
Given the following cluster definition:
The operator will inject an init container in the first instance of the cluster and the init container will start recovering the backup from the object storage.
When the recovery process is completed, the operator will start the instance to allow it to recover the transaction log files needed for the consistency of the restored data directory.
Once the recovery is complete, the operator will set the required superuser password into the instance. The new primary instance will start as usual, and the remaining instances will join the cluster as replicas.
The process is transparent for the user and it is managed by the instance manager running in the Pods.
You can optionally specify a
recoveryTarget to perform a point in time
recovery. If left unspecified, the recovery will continue up to the latest
available WAL on the default target timeline (
current for PostgreSQL up to
latest for version 12 and above).