Group Commit v4
The goal of Group Commit is to protect against data loss in case of single node failures or temporary outages. You achieve this by requiring more than one BDR node to successfully receive and confirm a transaction at COMMIT time.
During normal operation, Group Commit is completely transparent to the application. Upon failover, the reconciliation phase needs to be explicitly triggered or consolidated by either the application or a proxy in between. HARP provides native support for Group Commit and triggers the reconciliation phase, making this equally transparent to the client.
On the origin node, a transaction committed with Group Commit uses
two-phase commit underneath. Therefore, configure
high enough to handle all such transactions
originating per node.
To use Group Commit, first define a commit scope. This determines the BDR nodes involved in the commit of a transaction. Once a scope is established, you can configure a transaction to use Group Commit as follows:
The commit scope must be set before the transaction has written any data.
For this example, you might previously have defined the commit scope as:
This assumes a node group named
example_bdr_group exists and
includes at least two BDR nodes as members, either directly or in
subgroups. Any transaction committed in the
requires one extra confirmation from a BDR node in the group.
Together with the origin node, this accounts for "ANY 2" nodes out of
the group, on which the transaction is guaranteed to be visible and
durable after the commit.
Rules for commit scopes can depend on the node the transaction is committed on, that is, the node that acts as the origin for the transaction. To make this transparent for the application, BDR allows a commit scope to define different rules depending on where the transaction originates from.
For example, consider a EDB Postgres Distributed cluster with nodes spread across two data
centers: a left and a right one. Assume the top-level BDR node group
top_group. You can use the following commands to set up
subgroups and create a commit scope requiring all nodes in the local
data center to confirm the transaction but only one node from the
BDR nodes can send confirmations for a transaction at different points in time, similar to Commit At Most Once. In increasing levels of protection, from the perspective of the confirming node, these are:
received— A remote BDR node confirms the transaction immediately after receiving it, prior to starting the local application.
replicated— Confirm after applying changes of the transaction but before flushing them to disk.
durable— Confirm the transaction after all of its changes are flushed to disk.
visible(default) — Confirm the transaction after all of its changes are flushed to disk and it's visible to concurrent transactions.
In rules for commit scopes, you can append these confirmation levels
to the node group definition in parenthesis with
ON as follows:
ANY 2 (right_dc) ON replicated
ALL (left_dc) ON visible(default and may as well be omitted)
ALL (left_dc) ON received AND ANY 1 (right_dc) ON durable
For reference, the grammar for commit scopes is composed as follows:
While the grammar for
synchronous_standby_names and Commit
Scopes looks very similar, it is important to note that the former
does not account for the origin node, but the latter does.
Therefore, for example
synchronous_standby_names = 'ANY 1 (..)'
is equivalent to a Commit Scope of
ANY 2 (...). This choice
makes reasoning about majority easier and reflects that the origin
node also contributes to the durability and visibility of the
bdr.add_commit_scope creates a rule for the given
commit scope name and origin node group. If the rule is the same for
all nodes in the EDB Postgres Distributed cluster, invoking this function once for the
top-level node group is enough to fully define the commit scope.
Alternatively, you can invoke it multiple times with the same
commit_scope_name but different origin node groups and rules for
commit scopes that vary depending on the origin of the transaction.
To change a specific rule for a single origin node group in a
commit scope, you can use the function
You can use
bdr.remove_commit_scope to drop a single rule in
a commit scope. If you define multiple rules for the commit scope, you must invoke
this function once per rule to fully remove the entire
Removing a commit scope that is still used as default by a node group is not allowed