Release notes for EDB Postgres Distributed version 4.0.0 v4

Released: 01 Dec 2021

EDB Postgres Distributed version 4.0.0 contains BDR 34.0 and HARP 2.0. BDR 4.0 is a new major version of BDR and adopted with this release number is semantic versioning (for details see The two previous major versions are 3.7 and 3.6.

BDR4.0.0FeatureBDR on EDB Postgres Advanced 14 now supports following features which were previously only available on EDB Postgres Extended:
- Commit At Most Once - a consistency feature helping an application to commit each transaction only once, even in the presence of node failures
- Eager Replication - synchronizes between the nodes of the cluster before committing a transaction to provide conflict free replication
- Decoding Worker - separation of decoding into separate worker from wal senders allowing for better scalability with many nodes
- Estimates for Replication Catch-up times
- Timestamp-based Snapshots - providing consistent reads across multiple nodes for retrieving data as they appeared or will appear at a given time
- Automated dynamic configuration of row freezing to improve consistency of UPDATE/DELETE conflicts resolution in certain corner cases
- Assesment checks
- Support for handling missing partitions as conflicts rather than errors
- Advanced DDL Handling for NOT VALID constraints and ALTER TABLE
BDR4.0.0FeatureBDR on community version of PostgreSQL 12-14 now supports following features which were previously only available on EDB Postgres Advanced or EDB Postgres Extended:
- Conflict-free Replicated Data Types - additional data types which provide mathematically proven consistency in asynchronous multi-master update scenarios
- Column Level Conflict Resolution - ability to use per column last-update wins resolution so that UPDATEs on different fields can be "merged" without losing either of them
- Transform Triggers - triggers that are executed on the incoming stream of data providing ability to modify it or to do advanced programmatic filtering
- Conflict triggers - triggers which are called when conflict is detected, providing a way to use custom conflict resolution techniques
- CREATE TABLE AS replication
- Parallel Apply - allow multiple writers to apply the incoming changes
BDR4.0.0FeatureSupport streaming of large transactions.

This allows BDR to stream a large transaction (greater than logical_decoding_work_mem in size) either to a file on the downstream or to a writer process. This ensures that the transaction is decoded even before it's committed, thus improving parallelism. Further, the transaction can even be applied concurrently if streamed straight to a writer. This improves parallelism even more.

When large transactions are streamed to files, they are decoded and the decoded changes are sent to the downstream even before they are committed. The changes are written to a set of files and applied when the transaction finally commits. If the transaction aborts, the changes are discarded, thus wasting resources on both upstream and downstream.

Sub-transactions are also handled automatically.

This feature is available on PostgreSQL 14, EDB Postgres Extended 13+ and EDB Postgres Advanced 14, see Choosing a Postgres distribution appendix for more details on which features can be used on which versions of Postgres.

BDR4.0.0FeatureThe differences that existed in earlier versions of BDR between standard and enterprise edition have been removed. With BDR 4.0 there is one extension for each supported Postgres distribution and version, i.e., PostgreSQL v12-14, EDB Postgres Extended v12-14, and EDB Postgres Advanced 12-14.

Not all features are available on all versions of PostgreSQL, the available features are reported via feature flags using either bdr_config command line utility or bdr.bdr_features() database function. See Choosing a Postgres distribution for more details.

BDR4.0.0FeatureThere is no pglogical 4.0 extension that corresponds to the BDR 4.0 extension. BDR no longer has a requirement for pglogical.

This means also that only BDR extension and schema exist and any configuration parameters were renamed from pglogical. to bdr..

BDR4.0.0FeatureSome configuration options have change defaults for better post-install experience:
- Parallel apply is now enabled by default (with 2 writers). Allows for better performance, especially with streaming enabled.
- COPY and CREATE INDEX CONCURRENTLY are now streamed directly to writer in parallel (on Postgres versions where streaming is supported) to all available nodes by default, eliminating or at least reducing replication lag spikes after these operations.
- The timeout for global locks have been increased to 10 minutes
- The bdr.min_worker_backoff_delay now defaults to 1s so that subscriptions retry connection only once per second on error
BDR4.0.0FeatureGreatly reduced the chance of false positives in conflict detection during node join for table that use origin based conflict detection
BDR4.0.0FeatureMove configuration of CAMO pairs to SQL catalogs

To reduce chances of misconfiguration and make CAMO pairs within the BDR cluster known globally, move the CAMO configuration from the individual node's postgresql.conf to BDR system catalogs managed by Raft. This for example can prevent against inadvertently dropping a node that's still configured to be a CAMO partner for another active node.

Please see the Upgrades chapter for details on the upgrade process.

This deprecates GUCs bdr.camo_partner_of and bdr.camo_origin_for and replaces the functions bdr.get_configured_camo_origin_for() and get_configured_camo_partner_of with bdr.get_configured_camo_partner.

HARP2.0.0ChangeComplete rewrite of system in golang to optimize all operations
HARP2.0.0ChangeCluster state can now be bootstrapped or revised via YAML
HARP2.0.0FeatureConfiguration now in YAML, configuration file changed from harp.ini to config.yml
HARP2.0.0FeatureHARP Proxy deprecates need for HAProxy in supported architecture.

The use of HARP Router to translate DCS contents into appropriate online or offline states for HTTP-based URI requests meant a load balancer or HAProxy was necessary to determine the lead master. HARP Proxy now does this automatically without periodic iterative status checks.

HARP2.0.0FeatureUtilizes DCS key subscription to respond directly to state changes.

With relevant cluster state changes, the cluster responds immediately, resulting in improved failover and switchover times.

HARP2.0.0FeatureCompatibility with etcd SSL settings.

It is now possible to communicate with etcd through SSL encryption.

HARP2.0.0FeatureZero transaction lag on switchover.

Transactions are not routed to the new lead node until all replicated transactions are replayed, thereby reducing the potential for conflicts.

HARP2.0.0FeatureExperimental BDR Consensus layer.

Using BDR Consensus as the Distributed Consensus Service (DCS) reduces the amount of change needed for implementations.

HARP2.0.0FeatureExperimental built-in proxy.

Proxy implementation for increased session control.