EDB Docs - EDB Postgres Distributed (PGD) v6.5.0

Upgrade a PGD 5 cluster to PGD 6, moving from PGD Proxy to Connection Manager, with the Postgres version unchanged, using one of two approaches: full downtime or a rolling upgrade, either in place or by replacing nodes one at a time. A rolling upgrade starts by moving the cluster to Connection Manager, while every node is still on PGD 5.9.

Note

If you're also upgrading Postgres to a new minor version at the same time, see Upgrading Postgres and PGD together instead.

Note

This procedure assumes PGD Proxy runs on a PGD node. If PGD Proxy runs on a separate server, move it to a PGD node first, since Connection Manager only runs on PGD data nodes.

Before you begin

Confirm every node is running the supported starting version. See Supported PGD upgrade paths, then check the versions in your cluster:

pgd nodes list --versions

Upgrading with full downtime

Choose this approach when a maintenance window is acceptable and finishing quickly matters more than staying online. See Choosing between a rolling and full-downtime upgrade for the trade-offs against the two rolling approaches described below.

Disconnect applications from the cluster, and stop PGD Proxy on every node running it:
```
sudo systemctl stop pgd-proxy
```
Stop Postgres on every node:
```
sudo systemctl stop postgres
```
You can run this step one node at a time, but you must wait for each node to shut down cleanly before moving to the next. Monitor progress with sudo journalctl -u postgres -f.
Remove the PGD 5 CLI, proxy, and extension packages from each node once every node is stopped, then install the PGD 6 packages for your current Postgres version. Removal comes first since both versions target the same Postgres version and can't be installed side by side:
```
dnf remove edb-bdr5-<postgresversion> edb-pgd5-cli edb-pgd5-proxy
dnf install edb-pgd6-expanded-pg<postgres_version> -y
```
Set the configuration parameter bdr.enable_builtin_connection_manager to true on every node, before starting Postgres again, either in postgresql.conf or with ALTER SYSTEM. Since nothing is serving traffic yet, you don't need the separate restart-then-enable sequence a rolling upgrade uses to move the cluster onto Connection Manager first.
Start Postgres on every node:
```
systemctl start postgres
```
Each node upgrades to PGD 6 on start, with Connection Manager already enabled.
Confirm every node has rejoined the cluster and is healthy:
```
pgd nodes list
```
Every node should show ACTIVE under Join State and Up under Node Status.
Continue to Moving to Connection Manager.

Upgrading with a rolling upgrade

Upgrade nodes one at a time so applications stay available throughout. Move the whole cluster onto Connection Manager first, while every node is still on PGD 5.9, then upgrade each node either in place or by replacing it.

Moving to Connection Manager first

PGD 5.9 already includes Connection Manager, disabled by default. Before starting either approach below, move the whole cluster from PGD Proxy to Connection Manager, one node at a time, while every node is still running PGD 5.9.

Run the following query, as a database superuser connected to the PGD-enabled database on one of the nodes, to ensure that SCRAM hashes of all user passwords are the same across all nodes:

DO $$
DECLARE
    rec RECORD;
    command TEXT;    password TEXT;
BEGIN
    FOR rec IN SELECT rolname,rolpassword FROM pg_authid WHERE rolcanlogin = true AND rolpassword like 'SCRAM-SHA%'
    LOOP
        password := rec.rolpassword;
        command := 'ALTER ROLE ' || quote_ident(rec.rolname) || ' WITH ENCRYPTED PASSWORD ' || quote_literal(password);
        EXECUTE command;
    END LOOP;
END;
$$;
SELECT bdr.wait_slot_confirm_lsn(NULL, NULL);

Note

Don't add new users to 5.9 after executing this query. If you add any, run the query again. The block above doesn't change the passwords, it just ensures SCRAM hashes are the same across the cluster on all nodes.

For each node, one at a time:

Fence the node, so it doesn't become the write leader:
```
pgd node <node_name> set-option route_fence true
```
Set the configuration parameter bdr.enable_builtin_connection_manager to true:
```
ALTER SYSTEM SET bdr.enable_builtin_connection_manager = true;
```
Restart the server to pick up the configuration change:
```
sudo systemctl restart postgres
```
Stop PGD Proxy running on the server:
```
sudo systemctl stop pgd-proxy
```
Restart the server again:
```
sudo systemctl restart postgres
```
It starts with Connection Manager running on the default port. If the proxy read and write ports were different, you can change the Connection Manager read and write ports to match the proxy using bdr.alter_node_group_option().
Unfence the node:
```
pgd node <node_name> set-option route_fence false
```
It can now accept connections from the user and route to the write leader via Connection Manager.
Repeat for each remaining node, one at a time, until every node routes via Connection Manager.

Once every node is on Connection Manager, continue to Upgrading in place or Upgrading by replacing nodes.

Upgrading in place

This section only swaps the PGD 5 packages for PGD 6 on each node in turn, since PGD Proxy is already stopped everywhere once step 1 is done.

Move the whole cluster onto Connection Manager first. See Moving to Connection Manager first.

For each node, one at a time:

Check whether the node is the write leader for its group:
```
pgd group <group_name> show --summary
```
If it is, switch leadership to another node first:
```
pgd group <group_name> set-leader <new_leader_node_name>
```
Fence the node, so it doesn't become the write leader:
```
pgd node <node_name> set-option route_fence true
```
Stop Postgres on the node:
```
sudo systemctl stop postgres
```
Monitor progress with sudo journalctl -u postgres -f.
Remove the PGD 5 CLI, proxy, and extension packages, then install the PGD 6 packages for your current Postgres version. Removal comes first since both versions target the same Postgres version and can't be installed side by side:
```
dnf remove edb-bdr5-<postgresversion> edb-pgd5-cli edb-pgd5-proxy
dnf install edb-pgd6-expanded-pg<postgres_version> -y
```
Start Postgres. This step performs an in-place upgrade of the local node to PGD 6, with Connection Manager already routing for it:
```
sudo systemctl start postgres
```

Unfence the node:

pgd node <node_name> set-option route_fence false

Confirm the node is healthy and running the new version:
```
pgd nodes list --versions
```
Repeat for each remaining node, one at a time, until every node is running PGD 6.
Continue to Moving to Connection Manager below.

Upgrading by replacing nodes

Instead of upgrading the nodes, join new nodes that already have PGD 6 installed, then part and drop the existing PGD 5 nodes.

Move the existing cluster onto Connection Manager first. See Moving to Connection Manager first. PGD Proxy can't route to a PGD 6 node, so the cluster needs to already be routing through Connection Manager before a PGD 6 node joins.
Provision a new node with the PGD 6 packages for your current Postgres version already installed. Connection Manager is enabled on it by default.
Join the new node to the cluster using a logical join:
```
pgd node <node_name> setup --dsn "<new_node_dsn>" --cluster-dsn "<existing_node_dsn>" -D <pg_data>
```
Don't use bdr_init_physical, it requires the source and joining node to run the same PGD version, so it can't be used across a version change. See the pgd node setup command reference for the full set of options.
Wait for the new node to catch up with the cluster before routing traffic to it, then confirm it's healthy:
```
pgd nodes list
```
Update your application's connection configuration to include the new node once it's ready to take traffic.
Part one of the remaining PGD 5 nodes, then decommission it:
```
pgd node <node_name> part
```
Repeat for each remaining PGD 5 node, one at a time, until the whole cluster is on PGD 6.
Continue to Moving to Connection Manager below.

Moving to Connection Manager

Once every node is running PGD 6, regardless of which upgrade path you took, finish moving the cluster off PGD Proxy and onto Connection Manager.

Note

If you used either rolling upgrade approach, PGD Proxy is already stopped and applications already route through Connection Manager from the earlier step. Use the following as a final confirmation.

Confirm every node is running the new version:
```
pgd nodes list --versions
```
Confirm every node reports the same protocol_version in bdr.group_raft_details, confirming the whole cluster has moved to the new Raft protocol version, not just that individual nodes report success:
```
SELECT node_name, protocol_version FROM bdr.group_raft_details;
```
Confirm routing is enabled for every group:
```
pgd group <group_name> show --options
```
Then switch applications over to Connection Manager.
Stop any PGD Proxy services still running:
```
sudo systemctl stop pgd-proxy
```
Confirm the cluster is healthy:
```
pgd cluster show --health
```

Upgrading PGD 5 to PGD 6 v6.5.0

Note

Note

Before you begin

Upgrading with full downtime

Upgrading with a rolling upgrade

Moving to Connection Manager first

Note

Upgrading in place

Upgrading by replacing nodes

Moving to Connection Manager

Note

← Prev

↑ Up

Next →