Dynamic SQL-level configuration for BDR 0.9.0

The BDR team has recently introduced support for dynamically adding new nodes to a BDR group from SQL into the current development builds. Now no configuration file changes are required to add nodes and there’s no need to restart the existing or newly joining nodes.

This change does not appear in the current 0.8.0 stable release; it’ll land in 0.9.0 when that’s released, and can be found in the bdr-plugin/next branch in the mean time.

New nodes negotiate with the existing nodes for permission to join. Soon they’ll be able to the group without disrupting any DDL locking, global sequence voting, etc.

There’s also an easy node removal process so you don’t need to modify internal catalog tables and manually remove slots to drop a node anymore.

New node join process

With this change, the long-standing GUC-based configuration for BDR has been removed. bdr.connections no longer exists and you no longer configure connections with bdr.[conname]_dsn etc.

Instead, node addition is accomplished with the bdr.bdr_group_join(...) function. Because this is a function in the bdr extension, you must first CREATE EXTENSION bdr;. PostgreSQL doesn’t have extension dependencies and the bdr extension requires the btree_gist extension so you’ll have to CREATE EXTENSION btree_gist first.

Creating the first node

Creation of the first node must now be done explicitly using bdr.bdr_group_create. This promotes a standalone PostgreSQL database to a single-node BDR group, allowing other nodes to then be joined to it.

You must pass a node name and a valid externally-reachable connection string for the dsn parameter, e.g.:

CREATE EXTENSION btree_gist;

CREATE EXTENSION bdr;

SELECT bdr.bdr_group_join(
  local_node_name = 'node1',
  node_external_dsn := 'host=node1 dbname=mydb'
);

Note that the dsn is not used by the root node its self. It’s used by other nodes to connect to the root node, so you can’t use a dsn like host=localhost dbname=mydb if you intend to have nodes on multiple machines.

Adding other nodes

You can now join other nodes to form a fully functional BDR group by calling bdr.bdr_group_join and specifying a connection string that points to an existing node for the join_using_dsn. e.g.:

CREATE EXTENSION btree_gist;

CREATE EXTENSION bdr;

SELECT bdr.bdr_node_join(
    local_node_name := 'node2',
    node_external_dsn := 'host=node2 dbname=mydb',
    join_using_dsn := 'host=node1 dbname=mydb'
);

Here, node_external_dsn is an externally reachable connection string that can be used to establish connection to the new node, just like you supplied for the root node.

The join_using_dsn specifies the node that this new node should connect to when joining the group and establishing its membership. It won’t be used after joining.

Waiting until a node is ready

It’s now possible to tell when a new node has finished joining by calling bdr.node_join_wait(). This function blocks until the local node reports that it’s successfully joined a BDR group and is ready to execute commands.

Database name “bdr” now reserved

Additionally, the database name bdr is now reserved. It may not be used for BDR nodes, as BDR requires it for internal management. Hopefully this requirement can be removed later once a patch to the BGWorkers API has been applied to core.

Documentation moving into the source tree

The documentation on the PostgreSQL wiki sections for BDR is being converted into the same format as is used for PostgreSQL its self. It’s being added to the BDR extension source tree and will be available as part of the 0.9.0 release.

Trying it out

If you’d like to test out bdr-plugin/next, which is due to become BDR 0.9.0, take a look at the source install instructions and the quick-start guide.

There are no packages for BDR 0.9.0 yet, so if you try to install from packages you’ll just get 0.8.0.

Comments? Questions

Please feel free to leave comments and questions here, or post to pgsql-general with BDR-related questions.

We’re also now using GitHub to host a mirror of the BDR repository. We’re using the issue tracker there, so if you’ve found a bug and can supply a detailed report with the exact version and steps to reproduce, please file it there.