Removing nodes and groups v6.5.0

Removing a node from a PGD group

Since PGD is designed to recover from extended node outages, you must explicitly tell the system if you're removing a node permanently. If you permanently shut down a node and don't tell the other nodes, then performance suffers and eventually the whole system stops working.

Node removal, also called parting, is done using the bdr.part_node() function. You must specify the node name (as passed during node creation) to remove a node. You can call the bdr.part_node() function from any active node in the PGD group, including the node that you're removing.

Just like the join procedure, parting is done using Raft consensus and requires a majority of nodes to be online to work.

The parting process affects all nodes. The Raft leader manages a vote between nodes to see which node has the most recent data from the parting node. Then all remaining nodes make a secondary, temporary connection to the most recent node to allow them to catch up any missing data.

A parted node still is known to PGD but doesn't consume resources. A node might be added again under the same name as a parted node. In rare cases, you might want to clear all metadata of a parted node by using the function bdr.drop_node().

Automatically evicting lagging nodes

PGD retains the write-ahead log (WAL) a down or lagging node still needs, so the node can catch back up once it's available again. Retaining that WAL for too long can exhaust disk space on the healthy nodes.

Node eviction automates the parting of a lagging node once its replication lag crosses a configured threshold, instead of requiring a manual bdr.part_node() call. Configure eviction with two group options, set through bdr.alter_node_group_option():

• evict_node_action sets what happens once a node exceeds the lag threshold. none (the default) takes no action. part parts the node automatically.
• evict_node_lag_bytes sets the maximum lag, in bytes, a node can accumulate before eviction runs.

Set the maximum lag:

SELECT bdr.alter_node_group_option('<group_name>', 'evict_node_lag_bytes', '1000000000');

Set the eviction action:

SELECT bdr.alter_node_group_option('<group_name>', 'evict_node_action', 'part');

The write leader runs eviction checks, so the group needs routing enabled. Eviction also depends on Raft, so a majority of the cluster needs to stay reachable for it to run.

Removing a whole PGD group

PGD groups usually map to locations. When a location is no longer being deployed, it's likely that the PGD group for the location also needs to be removed.

The PGD group that's being removed must be empty. Before you can remove the group, you must part all the nodes in the group.