EDB Tutorial: How to Backup Your PostgreSQL to Azure Blob Storage with Barman

This tutorial is designed to guide you through how to setup a Barman backup to Azure Blob Storage for a PostgreSQL instance and store it in the cloud, using barman-cloud-backup.

For those who haven't used it before, Barman (Backup and Recovery Manager) is an open-source administration tool for remote backups and disaster recovery of PostgreSQL servers in business-critical environments.

To demonstrate the setting up Barman backup to Azure Blob Storage, we are using the below environment:

OS: RHEL 8

Database: EPAS

Database Version: 14.5

 

Step 1: Install Barman

 dnf -y install barman

Sample output:

[root@backuptest ~]# dnf -y install barman

Last metadata expiration check: 0:00:13 ago on Fri 27 Jan 2023 10:46:18 AM UTC.

Dependencies resolved.

============================================================================================================================================

 Package                        Architecture      Version                        Repository                                            Size

============================================================================================================================================

Installing:

 barman                         noarch            3.2.0-1.el8                    edb                                                   57 k

Installing dependencies:

 libpq5                         x86_64            15.1-42PGDG.rhel8              edb                                                  216 k

 python3-argcomplete            noarch            1.9.3-6.el8                    rhel-8-for-x86_64-appstream-eus-rhui-rpms             60 k

 python3-barman                 noarch            3.2.0-1.el8                    edb                                                  461 k

 python3-psycopg2               x86_64            2.8.6-1.rhel8                  edb                                                  178 k

 

Transaction Summary

============================================================================================================================================

Install  5 Packages

Total download size: 972 k

Installed size: 3.9 M

Downloading Packages:

(1/5): barman-3.2.0-1.el8.noarch.rpm                                                                        636 kB/s |  57 kB     00:00    

(2/5): libpq5-15.1-42PGDG.rhel8.x86_64.rpm                                                                  2.1 MB/s | 216 kB     00:00    

(3/5): python3-barman-3.2.0-1.el8.noarch.rpm                                                                4.0 MB/s | 461 kB     00:00    

(4/5): python3-psycopg2-2.8.6-1.rhel8.x86_64.rpm                                                            5.0 MB/s | 178 kB     00:00    

(5/5): python3-argcomplete-1.9.3-6.el8.noarch.rpm                                                           430 kB/s |  60 kB     00:00    

--------------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                       3.9 MB/s | 972 kB     00:00     

EnterpriseDB RPMs 8.6 - x86_64                                                                              1.7 MB/s | 1.7 kB     00:00    

Importing GPG key 0x7E30651C:

 Userid     : "EnterpriseDB Inc. (EnterpriseDB Yum Repositories) <packages@enterprisedb.com>"

 Fingerprint: CA40 9F7C 635F 2AE5 6C9E 8B34 E5ED E919 7E30 651C

 From       : /etc/pki/rpm-gpg/ENTERPRISEDB-GPG-KEY

Key imported successfully

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

  Preparing        :                                                                                                                    1/1 

  Installing       : python3-argcomplete-1.9.3-6.el8.noarch                                                                             1/5 

  Installing       : libpq5-15.1-42PGDG.rhel8.x86_64                                                                                    2/5 

  Running scriptlet: libpq5-15.1-42PGDG.rhel8.x86_64                                                                                    2/5 

  Installing       : python3-psycopg2-2.8.6-1.rhel8.x86_64                                                                              3/5 

  Installing       : python3-barman-3.2.0-1.el8.noarch                                                                                  4/5 

  Running scriptlet: barman-3.2.0-1.el8.noarch                                                                                          5/5 

  Installing       : barman-3.2.0-1.el8.noarch                                                                                          5/5 

  Running scriptlet: barman-3.2.0-1.el8.noarch                                                                                          5/5 

  Verifying        : barman-3.2.0-1.el8.noarch                                                                                          1/5 

  Verifying        : libpq5-15.1-42PGDG.rhel8.x86_64                                                                                    2/5 

  Verifying        : python3-barman-3.2.0-1.el8.noarch                                                                                  3/5 

  Verifying        : python3-psycopg2-2.8.6-1.rhel8.x86_64                                                                              4/5 

  Verifying        : python3-argcomplete-1.9.3-6.el8.noarch                                                                             5/5 

Installed products updated.

Installed:

  barman-3.2.0-1.el8.noarch                  libpq5-15.1-42PGDG.rhel8.x86_64                python3-argcomplete-1.9.3-6.el8.noarch         

  python3-barman-3.2.0-1.el8.noarch          python3-psycopg2-2.8.6-1.rhel8.x86_64         

 

Complete!

[root@backuptest ~]#

 

[root@backuptest ~]# barman --version

3.2.0 Barman by EnterpriseDB (www.enterprisedb.com)

[root@backuptest ~]#

 

Step 2: Install barman-cli-cloud

dnf install barman-cli-cloud
 

Sample output:

 

[root@backuptest ~]#  dnf install barman-cli-cloud

Last metadata expiration check: 0:01:09 ago on Fri 27 Jan 2023 10:46:18 AM UTC.

Dependencies resolved.

============================================================================================================================================

 Package                                 Architecture                  Version                             Repository                  Size

============================================================================================================================================

Installing:

 barman-cli-cloud                        noarch                        3.2.0-1.el8                         edb                         59 k

 

Transaction Summary

============================================================================================================================================

Install  1 Package

 

Total download size: 59 k

Installed size: 73 k

Is this ok [y/N]: y

Downloading Packages:

barman-cli-cloud-3.2.0-1.el8.noarch.rpm                                                                     362 kB/s |  59 kB     00:00    

--------------------------------------------------------------------------------------------------------------------------------------------

Total                                                                                                       355 kB/s |  59 kB     00:00     

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

  Preparing        :                                                                                                                    1/1 

  Installing       : barman-cli-cloud-3.2.0-1.el8.noarch                                                                                1/1 

  Running scriptlet: barman-cli-cloud-3.2.0-1.el8.noarch                                                                                1/1 

  Verifying        : barman-cli-cloud-3.2.0-1.el8.noarch                                                                                1/1 

Installed products updated.

Installed:

  barman-cli-cloud-3.2.0-1.el8.noarch                                                                                                       

Complete!

[root@backuptest ~]#

 

Step 3: Create a storage account on Azure  

Fill all the required details to create a storage account

 

 

 

 

 

Step 4: Create a container in your Azure storage account 

 

 

Step 5: Identify AZURE_STORAGE_CONNECTION_STRING and AZURE_STORAGE_KEY

We can find same from the storage account page, left panel under Security + networking > Access Keys

AZURE_STORAGE_KEY=> key

AZURE_STORAGE_CONNECTION_STRING=>connection key

 

 

 

Step 6: generate SAS for a container

Select the container created in step 4 and right click for drop down list to select “generate SAS” for  AZURE_STORAGE_SAS_TOKEN

AZURE_STORAGE_SAS_TOKEN => Blob SAS token

 

Step 7: Install azure-storage-blob

sudo pip3 install azure-storage-blob

Sample output:

[enterprisedb@backuptest ~]$ which pip3

/usr/bin/pip3

[enterprisedb@backuptest ~]$ which python3

/usr/bin/python3

[enterprisedb@backuptest ~]$ 

[enterprisedb@backuptest ~]$ sudo pip3 install azure-storage-blob

[sudo] password for enterprisedb: 

WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.

Collecting azure-storage-blob

  Downloading https://files.pythonhosted.org/packages/dd/fb/1501707ae8d921079ea826d16926e1b9b179d15264a2d239a08d7b374522/azure_storage_blob-12.13.1-py3-none-any.whl (377kB)

    100% |████████████████████████████████| 378kB 2.4MB/s 

Collecting msrest>=0.6.21 (from azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/15/cf/f2966a2638144491f8696c27320d5219f48a072715075d168b31d3237720/msrest-0.7.1-py3-none-any.whl (85kB)

    100% |████████████████████████████████| 92kB 9.7MB/s 

Collecting azure-core<2.0.0,>=1.23.1 (from azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/4f/43/18cac1c7c1c1c3ad9d0dc5f262bd5633f7b15a6b5f568dcb10a8ac8990aa/azure_core-1.24.2-py3-none-any.whl (178kB)

    100% |████████████████████████████████| 184kB 5.9MB/s 

Requirement already satisfied: cryptography>=2.1.4 in /usr/lib64/python3.6/site-packages (from azure-storage-blob)

Collecting requests-oauthlib>=0.5.0 (from msrest>=0.6.21->azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/6f/bb/5deac77a9af870143c684ab46a7934038a53eb4aa975bc0687ed6ca2c610/requests_oauthlib-1.3.1-py2.py3-none-any.whl

Requirement already satisfied: requests~=2.16 in /usr/lib/python3.6/site-packages (from msrest>=0.6.21->azure-storage-blob)

Collecting certifi>=2017.4.17 (from msrest>=0.6.21->azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/71/4c/3db2b8021bd6f2f0ceb0e088d6b2d49147671f25832fb17970e9b583d742/certifi-2022.12.7-py3-none-any.whl (155kB)

    100% |████████████████████████████████| 163kB 6.5MB/s 

Collecting isodate>=0.6.0 (from msrest>=0.6.21->azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/b6/85/7882d311924cbcfc70b1890780763e36ff0b140c7e51c110fc59a532f087/isodate-0.6.1-py2.py3-none-any.whl (41kB)

    100% |████████████████████████████████| 51kB 10.4MB/s 

Collecting typing-extensions>=4.0.1 (from azure-core<2.0.0,>=1.23.1->azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl

Requirement already satisfied: six>=1.11.0 in /usr/lib/python3.6/site-packages (from azure-core<2.0.0,>=1.23.1->azure-storage-blob)

Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/lib64/python3.6/site-packages (from cryptography>=2.1.4->azure-storage-blob)

Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-storage-blob)

  Downloading https://files.pythonhosted.org/packages/7e/80/cab10959dc1faead58dc8384a781dfbf93cb4d33d50988f7a69f1b7c9bbe/oauthlib-3.2.2-py3-none-any.whl (151kB)

    100% |████████████████████████████████| 153kB 7.0MB/s 

Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)

Requirement already satisfied: idna<2.8,>=2.5 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)

Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/lib/python3.6/site-packages (from requests~=2.16->msrest>=0.6.21->azure-storage-blob)

Requirement already satisfied: pycparser in /usr/lib/python3.6/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.1.4->azure-storage-blob)

Installing collected packages: oauthlib, requests-oauthlib, certifi, isodate, typing-extensions, azure-core, msrest, azure-storage-blob

Successfully installed azure-core-1.24.2 azure-storage-blob-12.13.1 certifi-2022.12.7 isodate-0.6.1 msrest-0.7.1 oauthlib-3.2.2 requests-oauthlib-1.3.1 typing-extensions-4.1.1

[enterprisedb@backuptest ~]$

 

Step 8: Export environment variables 

Export the below variables captured in step 5 and 6

export AZURE_STORAGE_CONNECTION_STRING=

export AZURE_STORAGE_KEY=

export AZURE_STORAGE_SAS_TOKEN=

 

Step 9: Execute backup

barman-cloud-backup --cloud-provider azure-blob-storage -J 4 https://barmanbackup.blob.core.windows.net/newcontainer prodbkp -v

Note: please use barman-cloud-backup --help to look for more option

 

Sample output:

On the backup completion:

2023-01-27 11:17:48,491 [11656] INFO: Backup end at LSN: 0/3000138 (000000010000000000000003, 00000138)

2023-01-27 11:17:48,491 [11656] INFO: Backup completed (start time: 2023-01-27 11:17:46.521570, elapsed time: 1 second)

2023-01-27 11:17:48,491 [11661] INFO: Upload process stopped (worker 0)

2023-01-27 11:17:48,491 [11662] INFO: Upload process stopped (worker 1)

2023-01-27 11:17:48,494 [11664] INFO: Upload process stopped (worker 3)

2023-01-27 11:17:48,494 [11663] INFO: Upload process stopped (worker 2)

[enterprisedb@backuptest ~]$

You can confirm the same from the portal

 

 

 

DISCLAIMER: 

 

• Please note that this sample is prepared and tested on the POC environment, you may need to change it according to your environment to make use of it.
• Please note that it is always advisable to test this in the lower environment before using it in the production environment.

 

Experience PostgreSQL peace of mind with Barman

Backup and recovery capabilities are essential to any database environment, especially one as complex and feature-rich as PostgreSQL. With Barman, you can rest assured that you have constant protection against data loss and a flexible backup strategy suited to your unique needs. Barman backup to Azure Blob Storage is just one of many ways to protect your most essential assets.

Read more of EDB's Barman documentation here.