Streaming replication with the WAL sender process v7

The changeset stream is accessible to the Replication Server publication server by the WAL sender process (walsender) using the streaming replication protocol.

The Replication Server publication server connects using the walsender interface through which changes are streamed on a continual basis. The continuous streaming eliminates the need for explicitly polling for changes.

The following are the basic synchronization steps using the log-based method:

  1. A streaming replication connection to the database server opens using libpq to establish a walsender communication channel.
  2. A separate thread monitors data changes streamed through the walsender interface.
  3. As the data changes become available, they transform to populate an in-memory cache.
  4. On the next scheduled interval, the in-memory cached data changes apply to each of the target databases in JDBC batches of SQL statements (referred to as transaction sets) in the same manner as described in Synchronization replication with the trigger-based method for the trigger-based method. If one or more target database servers aren't accessible, the data changes save in a local file on the host running the publication server. See In-memory caching and persistence for information on in-memory caching and data persistence.
  5. The value of the WAL segment’s log sequence number (LSN) identifying the last set of applied changes based on the last replicated transaction updates. The update is confirmed to the database server.
  6. The applied data changes clear from the in-memory cache.
  7. Steps 3 through 6 repeat.

A single SQL statement executed against a source table can result in many rows modified and returned in the changeset stream and, therefore, many SQL statements executed against the target table. For example, if a single UPDATE statement affects 10 rows in the source table, 10 rows are returned in the changeset stream, one for each row in the source table that was updated. When the publication server applies the changes to the target table, 10 UPDATE statements are executed.