System functions v6.4.0

Perform PGD management primarily by using functions you call from SQL. All functions in PGD are exposed in the bdr schema. Schema qualify any calls to these functions instead of putting bdr in the search_path.

Version information functions

bdr.bdr_version

This function retrieves the textual representation of the version of the BDR extension currently in use.

bdr.bdr_version_num

This function retrieves the version number of the BDR extension that is currently in use. Version numbers are monotonically increasing, allowing this value to be used for less-than and greater-than comparisons.

The following formula returns the version number consisting of major version, minor version, and patch release into a single numerical value:

MAJOR_VERSION * 10000 + MINOR_VERSION * 100 + PATCH_RELEASE

System information functions

bdr.get_relation_stats

Returns the relation information.

bdr.get_subscription_stats

Returns the current subscription statistics.

System and progress information parameters

PGD exposes some parameters that you can query directly in SQL using, for example, SHOW or the current_setting() function. You can also use PQparameterStatus (or equivalent) from a client application.

bdr.local_node_id

When you initialize a session, this is set to the node id the client is connected to. This allows an application to figure out the node it's connected to, even behind a transparent proxy.

It's also used with Connection pools and proxies.

bdr.last_committed_lsn

After every COMMIT of an asynchronous transaction, this parameter is updated to point to the end of the commit record on the origin node. Combining it with bdr.wait_for_apply_queue, allows applications to perform causal reads across multiple nodes, that is, to wait until a transaction becomes remotely visible.

transaction_id

If a CAMO transaction is in progress, transaction_id is updated to show the assigned transaction id. You can query this parameter only by using using PQparameterStatus or equivalent, and it isn't accessible in SQL. See Application use for a usage example.

Node status functions

bdr.is_node_connected

Synopsis

bdr.is_node_connected(node_name name)

Returns boolean by checking if the walsender for a given peer is active on this node.

bdr.is_node_ready

Synopsis

bdr.is_node_ready(node_name name, span interval DEFAULT NULL)

Returns boolean by checking if the lag is lower than the given span or lower than the timeout for TO ASYNC otherwise.

Consensus function

bdr.consensus_disable

Synopsis

bdr.consensus_disable(node_group_name text DEFAULT NULL)

Disables the consensus worker on the local node until server restart or until it's reenabled using bdr.consensus_enable (whichever happens first).

The node_group_name parameter is accepted for interface consistency but is currently a placeholder. The function disables the consensus worker database-wide, affecting all Raft instances on the node regardless of the value passed.

bdr.consensus_enable

Synopsis

bdr.consensus_enable(node_group_name text DEFAULT NULL)

Re-enables a disabled consensus worker on the local node.

The node_group_name parameter is accepted for interface consistency but is currently a placeholder. The function re-enables the consensus worker database-wide, affecting all Raft instances on the node regardless of the value passed.

bdr.consensus_proto_version

Returns currently used consensus protocol version by the local node.

Needed by the PGD group reconfiguration internal mechanisms.

bdr.consensus_snapshot_export

Synopsis

bdr.consensus_snapshot_export(version integer DEFAULT NULL, raft_instance_id integer DEFAULT NULL)

Generate a new PGD consensus snapshot from the currently committed-and-applied state of the local node and return it as bytea.

By default, a snapshot for the highest supported Raft version is exported. But you can override that by passing an explicit version number.

The raft_instance_id parameter identifies which Raft instance to export the snapshot from. When NULL (the default), the top-level Raft instance is used. To target a specific node group's Raft instance, pass its instance ID, which you can look up via bdr.raft_instances.

The exporting node doesn't have to be the current Raft leader, and it doesn't need to be completely up to date with the latest state on the leader. However, bdr.consensus_snapshot_import() might not accept such a snapshot.

The new snapshot isn't automatically stored to the local node's bdr.local_consensus_snapshot table. It's only returned to the caller.

The generated snapshot might be passed to bdr.consensus_snapshot_import() on any other nodes in the same PGD node group that's behind the exporting node's Raft log position.

The local PGD consensus worker must be disabled for this function to work. Typical usage is:

SELECT bdr.bdr_consensus_disable();
COPY (SELECT * FROM bdr.consensus_snapshot_export()) TO '/var/lib/postgresql/my_node_consensus_snapshot.data';
SELECT bdr.bdr_consensus_enable();

While the PGD consensus worker is disabled:

• DDL locking attempts on the node fail or time out.
• galloc sequences don't get new values.
• Eager and CAMO transactions pause or error.
• Other functionality that needs the distributed consensus system is disrupted. The required downtime is generally very brief.

Depending on the use case, it might be practical to extract a snapshot that already exists from the snapshot field of the bdr.local_consensus_snapshot table and use that instead. Doing so doesn't require you to stop the consensus worker.

bdr.consensus_snapshot_import

Synopsis

bdr.consensus_snapshot_import(snapshot bytea, raft_instance_id integer DEFAULT NULL, node_name bdr.regnode DEFAULT NULL)

Import a consensus snapshot that was exported by bdr.consensus_snapshot_export(), usually from another node in the same PGD node group.

It's also possible to use a snapshot extracted directly from the snapshot field of the bdr.local_consensus_snapshot table on another node.

This function is useful for resetting a PGD node's catalog state to a known good state in case of corruption or user error.

The raft_instance_id parameter identifies which Raft instance to import the snapshot into. When NULL (the default), the top-level Raft instance is used. To target a specific node group's Raft instance, pass its instance ID, which you can look up via bdr.raft_instances.

The node_name parameter attributes the snapshot to a specific source node, typically the leader node from which the snapshot was exported. When NULL (the default), no source node is attributed.

You can import the snapshot if the importing node's apply_index is less than or equal to the snapshot-exporting node's commit_index when the snapshot was generated. (See bdr.get_raft_status().) A node that can't accept the snapshot because its log is already too far ahead raises an error and makes no changes. The imported snapshot doesn't have to be completely up to date, as once the snapshot is imported the node fetches the remaining changes from the current leader.

The PGD consensus worker must be disabled on the importing node for this function to work. See notes on bdr.consensus_snapshot_export() for details.

It's possible to use this function to force the local node to generate a new Raft snapshot by running:

SELECT bdr.consensus_snapshot_import(bdr.consensus_snapshot_export());

This approach might also truncate the Raft logs up to the current applied log position.

bdr.consensus_snapshot_verify

Synopsis

bdr.consensus_snapshot_verify(snapshot bytea)

Verify the given consensus snapshot that was exported by bdr.consensus_snapshot_export(). The snapshot header contains the version with which it was generated and the node tries to verify it against the same version.

The snapshot might have been exported on the same node or any other node in the cluster. If the node verifying the snapshot doesn't support the version of the exported snapshot, then an error is raised.

bdr.get_consensus_status

Returns status information about the current consensus (Raft) worker.

bdr.get_raft_status

Returns status information about the current consensus (Raft) worker. Alias for bdr.get_consensus_status.

bdr.raft_leadership_transfer

Synopsis

bdr.raft_leadership_transfer(node_name text, 
                             wait_for_completion boolean, 
                             node_group_name text DEFAULT NULL)

Request the node identified by node_name to be the Raft leader. The request can be initiated from any of the PGD nodes and is internally forwarded to the current leader to transfer the leadership to the designated node. The designated node must be an ACTIVE PGD node with full voting rights.

If wait_for_completion is false, the request is served on a best-effort basis. If the node can't become a leader in the bdr.raft_global_lection_timeout period, then some other capable node becomes the leader again. Also, the leadership can change over the period of time per Raft protocol. A true return result indicates only that the request was submitted successfully.

If wait_for_completion is true, then the function waits until the given node becomes the new leader and possibly waits infinitely if the requested node fails to become Raft leader (for example, due to network issues). We therefore recommend that you always set a statement_timeout with wait_for_completion to prevent an infinite loop.

The node_group_name is optional and can be used to specify the name of the node group where the leadership transfer happens. If not specified, it defaults to NULL, which is interpreted as the top-level group in the cluster. If the node_group_name is specified, the function transfers leadership only within the specified node group.

Utility functions

bdr.transfer_all_leaderships

Transfers write and Raft leadership away from the local node on all groups where the local node currently holds write or Raft leadership. Use this function to drain all leaderships from a node before taking it offline.

Synopsis

bdr.transfer_all_leaderships(writeleader_timeout interval DEFAULT '10s')

Parameters

Return value

Returns true if all transfers succeeded or if the node held no leaderships, or false if any write leader transfer timed out.

If a write leader transfer times out, the function continues processing remaining write leaders and Raft transfers on a best-effort basis, and still returns false even if other transfers succeeded. Raft transfers are retried indefinitely until leadership changes or the query is cancelled, respecting the statement_timeout configuration parameter.

The function shares logic with bdr.routing_leadership_transfer and bdr.raft_leadership_transfer, completing both operations in a single call for faster results.

bdr.alter_node_kind

PGD5 introduced a concept of Task Manager Leader node. The node is selected by PGD, but for upgraded clusters, it's important to set the node_kind properly for all nodes in the cluster. Do this manually after upgrading to the latest PGD version by calling the bdr.alter_node_kind() SQL function for each node.

Synopsis

bdr.alter_node_kind(node_name text,
                    node_kind text);

Parameters

bdr.alter_subscription_skip_changes_upto

Because logical replication can replicate across versions, doesn't replicate global changes like roles, and can replicate selectively, sometimes the logical replication apply process can encounter an error and stop applying changes.

Wherever possible, fix such problems by making changes to the target side. CREATE any missing table that's blocking replication, CREATE a needed role, GRANT a necessary permission, and so on. But occasionally a problem can't be fixed that way and it might be necessary to skip entirely over a transaction. Changes are skipped as entire transactions—all or nothing. To decide where to skip to, use log output to find the commit LSN, per the example that follows, or peek the change stream with the logical decoding functions.

Unless a transaction made only one change, you often need to manually apply the transaction's effects on the target side, so it's important to save the problem transaction whenever possible, as shown in the examples that follow.

It's possible to skip over changes without bdr.alter_subscription_skip_changes_upto by using pg_catalog.pg_logical_slot_get_binary_changes to skip to the LSN of interest, so this is a convenience function. It does do a faster skip, although it might bypass some kinds of errors in logical decoding.

This function works only on disabled subscriptions.

The usual sequence of steps is:

1. Identify the problem subscription and LSN of the problem commit.
2. Disable the subscription.
3. Save a copy of the transaction using pg_catalog.pg_logical_slot_peek_changes on the source node, if possible.
4. bdr.alter_subscription_skip_changes_upto on the target node.
5. Apply repaired or equivalent changes on the target manually, if necessary.
6. Reenable the subscription.

Synopsis

  bdr.alter_subscription_skip_changes_upto(
    subname text,
    skip_upto_and_including pg_lsn
  );

Example

Apply of a transaction is failing with an error, and you've determined that lower-impact fixes such as changes on the target side can't resolve this issue. You determine that you must skip the transaction.

In the error logs, find the commit record LSN to skip to, as in this example:

ERROR:  XX000: CONFLICT: target_table_missing; resolver skip_if_recently_dropped returned an error: table does not exist
CONTEXT:  during apply of INSERT from remote relation public.break_me in xact with commit-end lsn 0/300AC18 xid 131315
committs 2021-02-02 15:11:03.913792+01 (action #2) (effective sess origin id=2 lsn=0/300AC18)
while consuming 'I' message from receiver for subscription bdr_regression_bdrgroup_node1_node2 (id=2667578509)
on node node2 (id=3367056606) from upstream node node1 (id=1148549230, reporiginid=2)

In this portion of log, you have the information you need: the_target_lsn: 0/300AC18 the_subscription: bdr_regression_bdrgroup_node1_node2

Next, disable the subscription so the apply worker doesn't try to connect to the replication slot:

  SELECT bdr.alter_subscription_disable('the_subscription');

You can't skip only parts of the transaction: it's all or nothing. So we strongly recommend that you save a record of it by copying it out on the provider side first, using the subscription's slot name.

  \copy (SELECT * FROM pg_catalog.pg_logical_slot_peek_changes('the_slot_name',
      'the_target_lsn', NULL, 'min_proto_version', '1', 'max_proto_version', '1',
      'startup_params_format', '1', 'proto_format', 'json'))
   TO 'transaction_to_drop.csv' WITH (FORMAT csv);

This example is broken into multiple lines for readability, but issue it in a single line. \copy doesn't support multi-line commands.

You can skip the change by changing peek to get, but bdr....skip_changes_upto does a faster skip that avoids decoding and outputting all the data:

  SELECT bdr.alter_subscription_skip_changes_upto('subscription_name',
      'the_target_lsn');

You can apply the same changes (or repaired versions of them) manually to the target node, using the dumped transaction contents as a guide.

Finally, reenable the subscription:

  SELECT bdr.alter_subscription_enable('the_subscription');

bdr.connection_manager_refresh_pools

This function tells the PGD connection manager to refresh its connection pools.

This function is useful for integration with other tools or when an immediate re-evaluation of connection pools is required. This function operates only on the local node; to refresh connection managers across the entire cluster, you must execute the function on each node individually. Execution requires the bdr_superuser role. Expand commentComment on line R861Resolved

Synopsis

SELECT bdr.connection_manager_refresh_pools();

bdr.get_node_sub_apply_lsn

You can use this function on a subscriber to get the last LSN that was received and applied from the given origin.

Synopsis

bdr.get_node_sub_apply_lsn(node_name name)

Parameters

bdr.get_node_sub_receive_lsn

You can use this function on a subscriber to get the last LSN that was received from the given origin. It can be either unfiltered or filtered to take into account only relevant LSN increments for transactions to be applied.

The difference between the output of this function and the output of bdr.get_node_sub_apply_lsn() measures the size of the corresponding apply queue.

Synopsis

bdr.get_node_sub_receive_lsn(node_name name, committed bool default true)

Parameters

bdr.global_lock_table

This function acquires a global DML locks on a given table. See DDL locking details for information about global DML lock.

Synopsis

bdr.global_lock_table(relation regclass)

Parameters

Notes

This function acquires the global DML lock independently of the ddl_locking setting.

The bdr.global_lock_table function requires UPDATE, DELETE, or TRUNCATE privilege on the locked relation unless bdr.backwards_compatibility is set to 30618 or lower.

Example

pgddb=# SELECT bdr.local_group_slot_name();
 local_group_slot_name
-----------------------
 bdr_pgddb_bdrgroup

bdr.group_lease_override

Manually transfers or cancels node group lease ownership. Under normal operating conditions, PGD nodes transfer leases automatically. However, if a lease holder crashes, stops, or becomes disconnected from consensus, other nodes cannot acquire the lease until the current lease expires. This function bypasses that waiting period.

Parameters

bdr.local_group_slot_name

Returns the name of the group slot on the local node.

bdr.node_group_type

Returns the type of the given node group. Returned value is the same as what was passed to bdr.create_node_group() when the node group was created, except global is returned if the node_group_type was passed as NULL when the group was created.

Example

pgddb=# SELECT bdr.node_group_type('bdrgroup');
 node_group_type
-----------------
 global

bdr.replicate_ddl_command

Function to replicate a DDL command to a group of nodes.

Synopsis

bdr.replicate_ddl_command(ddl_cmd text,
                          replication_sets text[],
                          ddl_locking text,
                          execute_locally bool)

Parameters

Notes

The only required parameter of this function is ddl_cmd.

bdr.replicate_ddl_command() always replicates the command and is unaffected by the setting of bdr.ddl_replication .

bdr.run_on_all_nodes

Function to run a query on all nodes.

Synopsis

bdr.run_on_all_nodes(query text)

Parameters

Notes

This function connects to other nodes and executes the query, returning a result from each of them in JSON format. Multiple rows might be returned from each node, encoded as a JSON array. Any errors, such as being unable to connect because a node is down, are shown in the response field. No explicit statement_timeout or other runtime parameters are set, so defaults are used.

This function doesn't go through normal replication. It uses direct client connection to all known nodes. By default, the connection is created with bdr.ddl_replication = off, since the commands are already being sent to all of the nodes in the cluster.

In PGD 6 and later, this function also sets bdr.xact_replication=off on the connection to ensure that transaction run locally only when the command is executed on another node.

Be careful when using this function since you risk breaking replication and causing inconsistencies between nodes. Use either transparent DDL replication or bdr.replicate_ddl_command() to replicate DDL. DDL might be blocked in a future release.

Example

It's useful to use this function in monitoring, for example, as in the following query:

SELECT bdr.run_on_all_nodes($$
	SELECT local_slot_name, origin_name, target_name, replay_lag_size
      FROM bdr.node_slots
     WHERE origin_name IS NOT NULL
$$);

This query returns something like this on a two-node cluster:

[
    {
        "dsn": "host=node1 port=5432 dbname=pgddb user=postgres ",
        "node_id": "2232128708",
        "response": {
            "command_status": "SELECT 1",
            "command_tuples": [
                {
                    "origin_name": "node1",
                    "target_name": "node2",
                    "local_slot_name": "bdr_pgddb_bdrgroup_node2",
                    "replay_lag_size": "0 bytes"
                }
            ]
        },
        "node_name": "node1"
    },
    {
        "dsn": "host=node2 port=5432 dbname=pgddb user=postgres ",
        "node_id": "2058684375",
        "response": {
            "command_status": "SELECT 1",
            "command_tuples": [
                {
                    "origin_name": "node2",
                    "target_name": "node1",
                    "local_slot_name": "bdr_pgddb_bdrgroup_node1",
                    "replay_lag_size": "0 bytes"
                }
            ]
        },
        "node_name": "node2"
    }
]

bdr.run_on_group

Function to run a query on a group of nodes.

Synopsis

bdr.run_on_group(node_group_name text, query text)

Parameters

Notes

This function connects to other nodes and executes the query, returning a result from each of them in JSON format. Multiple rows can be returned from each node, encoded as a JSON array. Any errors, such as being unable to connect because a node is down, are shown in the response field. No explicit statement_timeout or other runtime parameters are set, so defaults are used.

This function doesn't go through normal replication. It uses direct client connection to all known nodes. By default, the connection is created with bdr.ddl_replication = off to avoid replication issues when the same replicated DDL command is sent to multiple nodes.

In PGD 6 and later, this function also sets bdr.xact_replication=off on the connection to ensure that transactions run locally only when the command is executed on another node.

Be careful when using this function since you risk breaking replication and causing inconsistencies between nodes in the group. For global schema changes, to replicate DDL, use either transparent DDL replication or bdr.replicate_ddl_command().

bdr.run_on_nodes

Function to run a query on a specified list of nodes.

Synopsis

bdr.run_on_nodes(node_names text[], query text)

Parameters

Notes

This function connects to other nodes and executes the query, returning a result from each of them in JSON format. Multiple rows can be returned from each node, encoded as a JSON array. Any errors, such as being unable to connect because a node is down, are shown in the response field. No explicit statement_timeout or other runtime parameters are set, so defaults are used.

This function doesn't go through normal replication. It uses direct client connection to all known nodes. By default, the connection is created with bdr.ddl_replication = off to avoid replication issues when the same replicated DDL command is sent to multiple nodes.

In PGD 6 and later, this function also sets bdr.xact_replication=off on the connection to ensure that transactions run locally only when the command is executed on another node.

Be careful when using this function since you risk breaking replication and causing inconsistencies between nodes. For global schema changes, to replicate DDL, use either transparent DDL replication or bdr.replicate_ddl_command().

bdr.stat_reset

Resets statistics for the specified category. Running this function clears the value of the column stats_reset in the bdr.stat_reconcile_commit_scope view.

Requires bdr_superuser privileges.

Parameters

bdr.wait_for_apply_queue

The function bdr.wait_for_apply_queue allows a PGD node to wait for the local application of certain transactions originating from a given PGD node. It returns only after all transactions from that peer node are applied locally. An application or a proxy can use this function to prevent stale reads.

For convenience, PGD provides a variant of this function for CAMO and the CAMO partner node. See bdr.wait_for_camo_partner_queue .

In case a specific LSN is given, that's the point in the recovery stream from which the peer waits. You can use this with bdr.last_committed_lsn retrieved from that peer node on a previous or concurrent connection.

If the given target_lsn is NULL, this function checks the local receive buffer and uses the LSN of the last transaction received from the given peer node, effectively waiting for all transactions already received to be applied. This is especially useful in case the peer node has failed and it's not known which transactions were sent. In this case, transactions that are still in transit or buffered on the sender side aren't waited for.

Synopsis

bdr.wait_for_apply_queue(peer_node_name TEXT, target_lsn pg_lsn)

Parameters

bdr.wait_for_xid_progress

You can use this function to wait for the given transaction (identified by its XID) originated at the given node (identified by its node id) to make enough progress on the cluster. The progress is defined as the transaction being applied on a node and this node having seen all other replication changes done before the transaction is applied.

Synopsis

bdr.wait_for_xid_progress(origin_node_id oid, origin_topxid int4, allnodes boolean DEFAULT true)

Parameters

Notes

You can use the function only for those transactions that replicated a DDL command because only those transactions are tracked currently. If a wrong origin_node_id or origin_topxid is supplied, the function might wait forever or until statement_timeout occurs.

bdr.wait_node_confirm_lsn

Wait until a node passes a certain LSN.

This function allows you to wait until the last write on this session was replayed to one or all nodes.

Upon being called, the function waits for a node to pass a certain LSN. If no LSN is supplied, the current wal_flush_lsn (using the pg_current_wal_flush_lsn() function) position is used on the local node. Supplying a node name parameter tells the function to wait for that node to pass the LSN. If no node name is supplied (by passing NULL), the function waits until all the nodes pass the LSN.

We recommend using this function if you are using Optimized topology instead of bdr.wait_slot_confirm_lsn .

This is because in an optimized topology, not all nodes have replication slots, so the function bdr.wait_slot_confirm_lsn might not work as expected. bdr.wait_node_confirm_lsn is designed to work with nodes that don't have replication slots, using alternative strategies to determine the progress of a node.

If a node is currently down, isn't updating, or simply can't be connected to, the wait will continue indefinitely. To avoid this condition, set the statement_timeout to the maximum amount of time you are prepared to wait.

Synopsis

bdr.wait_node_confirm_lsn(node_name text DEFAULT NULL, target_lsn pg_lsn DEFAULT NULL)

Parameters

Notes

Requires bdr_application privileges to use.

bdr.wait_slot_confirm_lsn

Allows you to wait until the last write on this session was replayed to one or all nodes.

Waits until a slot passes a certain LSN. If no position is supplied, the current write position is used on the local node.

If no slot name is passed, it waits until all PGD slots pass the LSN.

The function polls every 1000 ms for changes from other nodes.

If a slot is dropped concurrently, the wait ends for that slot. If a node is currently down and isn't updating its slot, then the wait continues. You might want to set statement_timeout to complete earlier in that case.

If you are using Optimized topology , we recommend using bdr.wait_node_confirm_lsn instead.

Synopsis

bdr.wait_slot_confirm_lsn(slot_name text DEFAULT NULL, target_lsn pg_lsn DEFAULT NULL)

Notes

Requires bdr_application privileges to use.

Parameters

Global advisory locks

PGD supports global advisory locks. These locks are similar to the advisory locks available in PostgreSQL except that the advisory locks supported by PGD are global. They follow semantics similar to DDL locks. So an advisory lock is obtained by majority consensus and can be used even if one or more nodes are down or lagging behind, as long as a majority of all nodes can work together.

Currently only EXCLUSIVE locks are supported. So if another node or another backend on the same node has already acquired the advisory lock on the object, then other nodes or backends must wait for the lock to be released.

Advisory lock is transactional in nature. So the lock is released when the transaction ends unless you explicitly release it before the end of the transaction. In this case, it becomes available as soon as it's released. Session-level advisory locks aren't currently supported.

Global advisory locks are reentrant. So if the same resource is locked three times, you must then unlock it three times to release it for use in other sessions.

bdr.global_advisory_lock

This function acquires an EXCLUSIVE lock on the provided object. If the lock isn't available, then it waits until the lock becomes available or the bdr.global_lock_timeout is reached.

Synopsis

bdr.global_advisory_lock(key bigint)

parameters

• key — The object on which an advisory lock is acquired.

Synopsis

bdr.global_advisory_lock(key1 integer, key2 integer)

Parameters

bdr.global_advisory_unlock

This function releases a previously acquired lock on the application-defined source. The lock must have been obtained in the same transaction by the application. Otherwise, an error is raised.

Synopsis

bdr.global_advisory_unlock(key bigint)

Parameters

Synopsis

bdr.global_advisory_unlock(key1 integer, key2 integer)

Parameters

Monitoring functions

bdr.monitor_group_versions

To provide a cluster-wide version check, this function uses PGD version information returned from the view bdr.group_version_details.

Synopsis

bdr.monitor_group_versions()

Notes

This function returns a record with fields status and message, as explained in Monitoring.

This function calls bdr.run_on_all_nodes().

bdr.monitor_group_raft

To provide a cluster-wide Raft check, this function uses PGD Raft information returned from the view bdr.group_raft_details.

Synopsis

bdr.monitor_group_raft()

Parameters

Notes

This function returns a record with fields status and message, as explained in Monitoring.

This function calls bdr.run_on_all_nodes().

bdr.monitor_local_replslots

This function uses replication slot status information returned from the view pg_replication_slots (slot active or inactive) to provide a local check considering all replication slots except the PGD group slots.

This function also provides status information on subscriber-only nodes that are operating as subscriber-only group leaders in a PGD cluster when optimized topology is enabled.

Synopsis

bdr.monitor_local_replslots()

Notes

This function returns a record with fields status and message.

Further explaination is available in Monitoring replication slots.

bdr.wal_sender_stats

If the decoding worker is enabled, this function shows information about the decoder slot and current logical change record (LCR) segment file being read by each WAL sender.

Synopsis

bdr.wal_sender_stats()

Output columns

bdr.get_decoding_worker_stat

If the decoding worker is enabled, this function shows information about the state of the decoding worker associated with the current database. This also provides more granular information about decoding worker progress than is available via pg_replication_slots.

Synopsis

bdr.get_decoding_worker_stat()

Output columns

Notes

For details, see Monitoring WAL senders using LCR.

bdr.lag_control

If Lag Control is enabled, this function shows information about the commit delay and number of nodes conforming to their configured lag measure for the local node and current database.

Synopsis

bdr.lag_control()

Output columns

Routing functions

bdr.routing_leadership_transfer

Changing the routing leader transfers the leadership of the node group to another node. If leader_name is NULL, the system automatically selects the most appropriate node based on write leader progress.

Synopsis

bdr.routing_leadership_transfer(node_group_name text,
              leader_name text DEFAULT NULL,
              transfer_method text DEFAULT 'strict',
              transfer_timeout interval DEFAULT '10s');

Parameters

Notes

When leader_name is NULL, the function selects the best candidate node by identifying the node with the furthest write leader progress and retrying selection until a suitable candidate is available or the timeout is reached. Automatic selection is useful for automated failover scenarios where manual node selection isn't required. When using Quorum Commit as the commit scope kind, pass NULL for leader_name to trigger automatic selection of the most appropriate node.

Selecting a candidate automatically

During automatic leader selection (when leader_name is NULL):

• Route-fenced nodes (nodes with route_fence enabled via bdr.alter_node_option) aren't considered candidates for leadership transfer.
• Nodes with route_writes set to false aren't considered candidates for leadership transfer.
• The route_priority option doesn't influence automatic selection. Selection is based solely on write leader progress among eligible nodes.

Running in a mixed-version cluster

Automatic leader selection (passing NULL for leader_name) requires PGD 6.4 and Raft protocol version 6004 or above.

When calling the function on pre-6.4 nodes, the function exits without performing any action, as the older function signature requires a non-NULL leader_name parameter.

When calling the function on 6.4 nodes, the behavior depends on the Raft protocol version:

• If Raft protocol version is below 6004 and a PGD 6.4 or above node is the Raft leader, the function returns ERROR: leadership auto transfer is supported from raft protocol version 6004 and above.
• If a pre-6.4 node is the Raft leader, the operation times out and returns ERROR: switchover timed out.

For reliable automatic leader selection, ensure all nodes in the cluster are running PGD 6.4 or later with Raft protocol version 6004 or above.

CAMO functions

CAMO requires that a client actively participates in the committing of a transaction by following the transactions progress. The functions listed here are used for that purpose and explained in CAMO.

bdr.is_camo_partner_connected

Allows checking of the connection status of a CAMO partner node configured in pair mode. There currently is no equivalent for CAMO used with eager replication.

Synopsis

bdr.is_camo_partner_connected()

Return value

A Boolean value indicating whether the CAMO partner is currently connected to a WAL sender process on the local node and therefore can receive transactional data and send back confirmations.

bdr.is_camo_partner_ready

Allows checking of the readiness status of a CAMO partner node configured in pair mode. Underneath, this triggers the switch to and from local mode.

Synopsis

bdr.is_camo_partner_ready()

Return value

A Boolean value indicating whether the CAMO partner can reasonably be expected to confirm transactions originating from the local node in a timely manner, that is, before timeout for TO ASYNC expires.

bdr.get_configured_camo_partner

This function shows the local node's CAMO partner (configured by pair mode).

Synopsis

bdr.get_configured_camo_partner()

bdr.wait_for_camo_partner_queue

The function is a wrapper around bdr.wait_for_apply_queue defaulting to query the CAMO partner node. It returns an error if the local node isn't part of a CAMO pair.

Synopsis

bdr.wait_for_camo_partner_queue()

bdr.camo_transactions_resolved

This function begins a wait for CAMO transactions to be fully resolved.

Synopsis

bdr.camo_transactions_resolved()

bdr.logical_transaction_status

To check the status of a transaction that was being committed when the node failed, the application must use this function, passing as parameters the node id of the node the transaction originated from and the transaction id on the origin node.

Synopsis

bdr.logical_transaction_status(node_id OID, xid OID,
                               require_camo_partner boolean DEFAULT true)

Parameters

Return value

The function returns one of these results:

• 'committed'::TEXT — The transaction was committed, is visible on both nodes of the CAMO pair, and is eventually replicated to all other PGD nodes. No need for the client to retry it.

• 'aborted'::TEXT — The transaction was aborted and isn't replicated to any other PGD node. The client needs to either retry it or escalate the failure to commit the transaction.

• 'in progress'::TEXT — The transaction is still in progress on this local node and wasn't committed or aborted yet. The transaction might be in the COMMIT phase, waiting for the CAMO partner to confirm or deny the commit. The recommended client reaction is to disconnect from the origin node and reconnect to the CAMO partner to query that instead. With a load balancer or proxy in between, where the client lacks control over which node gets queried, the client can only poll repeatedly until the status switches to either 'committed' or 'aborted'.

  For eager all-node replication, peer nodes yield this result for transactions that aren't yet committed or aborted. Even transactions not yet replicated (or not even started on the origin node) might yield an in progress result on a peer PGD node in this case. However, the client must not query the transaction status prior to attempting to commit on the origin.

• 'unknown'::TEXT — The transaction specified is unknown because it's either in the future, not replicated to that specific node yet, or too far in the past. The status of such a transaction isn't yet or is no longer known. This return value is a sign of improper use by the client.

The client must be prepared to retry the function call on error.

Commit Scope functions

bdr.add_commit_scope

Deprecated. Use bdr.create_commit_scope instead. Previously, this function was used to add a commit scope to a node group. It's now deprecated and will emit a warning until it is removed in a future release, at which point it will raise an error.

bdr.create_commit_scope

bdr.create_commit_scope creates a rule for the given commit scope name and origin node group. If the rule is the same for all nodes in the EDB Postgres Distributed cluster, invoking this function once for the top-level node group is enough to fully define the commit scope.

Alternatively, you can invoke it multiple times with the same commit_scope_name but different origin node groups and rules for commit scopes that vary depending on the origin of the transaction.

Synopsis

bdr.create_commit_scope(
    commit_scope_name NAME,
    origin_node_group NAME,
    rule TEXT,
    wait_for_ready boolean DEFAULT true)

Note

bdr.create_commit_scope replaces the deprecated bdr.add_commit_scope function. Unlike add_commit_scope, it doesn't silently overwrite existing commit scopes when the same name is used. Instead, an error is reported.

bdr.alter_commit_scope

bdr.alter_commit_scope allows you to change a specific rule for a single origin node group in a commit scope.

Synopsis

bdr.alter_commit_scope(
    commit_scope_name NAME,
    origin_node_group NAME,
    rule TEXT)

bdr.drop_commit_scope

Drops a single rule in a commit scope. If you define multiple rules for the commit scope, you must invoke this function once per rule to fully remove the entire commit scope.

Synopsis

bdr.drop_commit_scope(
    commit_scope_name NAME,
    origin_node_group NAME)

bdr.remove_commit_scope

Deprecated. Use bdr.drop_commit_scope instead. Previously, this function was used to remove a commit scope from a node group. It's now deprecated and will emit a warning until it is removed in a future release, at which point it will raise an error.

Apply error policy functions

These functions let you review, resolve, and analyze replication failures recorded by the apply error policy. See the apply_error_policy, apply_error_max_retries, apply_error_max_skips, and apply_error_max_record_size options in bdr.alter_node_group_option for configuration details. For a step-by-step guide to configuring a policy and resolving failures, see Handling replication apply errors.

bdr.skip_failed_change

Marks a single captured change in bdr.pending_failed_changes as skipped. The change status is updated to skipped but the row isn't deleted.

Synopsis

bdr.skip_failed_change(change_id bigint)

Parameters

bdr.skip_failed_transaction

Marks all pending changes in a failed transaction as skipped and optionally re-enables the subscription.

Synopsis

bdr.skip_failed_transaction(
    subscription_name name,
    remote_xid xid,
    remote_commit_lsn pg_lsn,
    dry_run boolean DEFAULT false,
    re_enable boolean DEFAULT false
)

bdr.skip_failed_transaction(
    failed_txn_id uuid,
    dry_run boolean DEFAULT false,
    re_enable boolean DEFAULT false
)

Parameters

Return value

Returns the number of changes skipped. Returns 0 if the transaction was already resolved (no error is raised).

bdr.apply_failed_change

Retries applying a single captured change.

Synopsis

bdr.apply_failed_change(change_id bigint)

Parameters

bdr.apply_failed_transaction

Retries all pending changes for a failed transaction. Defaults to dry-run mode so you can preview the outcome before committing.

Synopsis

bdr.apply_failed_transaction(
    subscription_name name,
    remote_xid xid,
    remote_commit_lsn pg_lsn,
    dry_run boolean DEFAULT true,
    re_enable boolean DEFAULT false
)

bdr.apply_failed_transaction(
    failed_txn_id uuid,
    dry_run boolean DEFAULT true,
    re_enable boolean DEFAULT false
)

Parameters

Return value

Returns one row per change with the following columns.

If an error occurs during apply, the function returns results for all changes up to and including the failed one. If the transaction was already resolved, the function returns 0 rows without raising an error.

bdr.modify_failed_change

Edits the column values of a single captured change before retrying the change. Updates the modified_values column in bdr.pending_failed_changes and sets the change status to modified.

Synopsis

bdr.modify_failed_change(change_id bigint, new_values jsonb)

Parameters

bdr.reconstruct_change_sql

Generates executable SQL for a single captured change. The bdr.pending_failed_changes_sql view uses this function internally.

Synopsis

bdr.reconstruct_change_sql(
    operation text,
    schema_name name,
    table_name name,
    key_columns jsonb,
    old_values jsonb,
    new_values jsonb
)

Parameters

Return value

Returns a text string containing the reconstructed SQL statement. DDL operations include a SET search_path prefix when a search path was in effect during the original execution.

bdr.reconstruct_failed_transaction

Generates a complete SQL script for all captured changes in a failed transaction, wrapped in BEGIN and COMMIT.

Synopsis

bdr.reconstruct_failed_transaction(
    subscription_name name,
    remote_xid xid,
    remote_commit_lsn pg_lsn
)

bdr.reconstruct_failed_transaction(failed_txn_id uuid)

Parameters

Return value

Returns a text SQL script containing all changes in the transaction. Use the script to review or manually replay the complete transaction.

bdr.resolve_failed_transaction

Marks all pending changes in a failed transaction as resolved, advances the subscription's position past the failed transaction, and optionally re-enables the subscription. Defaults to dry-run mode so you can preview the outcome before committing.

Synopsis

bdr.resolve_failed_transaction(
    subscription_name name,
    remote_xid xid,
    remote_commit_lsn pg_lsn,
    resolution text DEFAULT 'skipped',
    re_enable boolean DEFAULT false,
    dry_run boolean DEFAULT true
)

bdr.resolve_failed_transaction(
    failed_txn_id uuid,
    resolution text DEFAULT 'skipped',
    re_enable boolean DEFAULT false,
    dry_run boolean DEFAULT true
)

Parameters

Return value

In dry-run mode, returns a text report describing the planned changes. When dry_run is false, returns a text confirmation of the resolution applied.

Notes

Calling this function also inserts a summary record into bdr.resolved_transactions. Once resolved, you can truncate bdr.pending_failed_changes to reclaim space while retaining a history of resolutions.

If the transaction was already resolved, the function raises an error with details about the prior resolution.

bdr.analyze_skip_impact

Analyzes the safety and potential data impact of skipping a single captured change. Returns a risk assessment based on the error type, operation, and optionally the current local database state.

Synopsis

bdr.analyze_skip_impact(
    change_id bigint,
    verify_state boolean DEFAULT true,
    include_transaction_analysis boolean DEFAULT true
)

Parameters

Return value

Returns a row with the following columns.

Safety classifications

bdr.analyze_transaction_skip_impact

Analyzes the skip safety for all pending changes in a failed transaction, returning one row per change.

Synopsis

bdr.analyze_transaction_skip_impact(
    subscription_name name,
    remote_xid xid,
    remote_commit_lsn pg_lsn
)

Parameters

Return value

Returns one row per change in the transaction, with the following columns.

bdr.format_skip_analysis

Returns a formatted, human-readable analysis of the skip safety for a single captured change.

Synopsis

bdr.format_skip_analysis(change_id bigint)

Parameters

Return value

Returns a text report including the safety classification, recommendation, explanation, and any state verification results.

bdr.alter_subscription_reset_error_counters

Resets the error retry and skip counters for a subscription to their initial values. Use the reset after resolving replication failures to allow the subscription's error policy thresholds to apply fresh.

Synopsis

bdr.alter_subscription_reset_error_counters(subscription_name name)

Parameters