Trusted Postgres Architect 23.1 to 23.11 release notes v23

Suggest edits

TPA 23.11

Released: 2023-01-31

Notable changes

  • TPA-180 Introduce experimental support for PGD-Always-ON architecture (to be released later this year). PGD-Always-ON architecture will use the upcoming BDR version 5. Initial support has been added for internal purposes and will be improved in upcoming releases.

Minor changes

  • TPA-349 Bump dependency versions Bump cryptography version from 38.0.4 to 39.0.0 Bump jq version from 1.3.0 to 1.4.0
  • TPA-345 Change TPAexec references to TPA in documentation. Update the documentation to use 'TPA' instead of 'TPAexec' when referring to the product.

TPA 23.10

Released: 2023-01-04

Minor changes

  • TPA-161 Introduce harp_manager_restart_on_failure setting (defaults to false) to enable process restart on failure for the harp-manager systemd service

Bug Fixes

  • TPA-281 Delete FMS security groups when deprovisioning an AWS cluster Fixes a failure to deprovision a cluster's VPC because of unremoved dependencies.
  • TPA-305 Add enterprisedb_password to pre-generated secrets for Tower
  • TPA-306 Prefer PEM_PYTHON_EXECUTABLE, if present, to /usr/bin/python3 Fixes a Python module import error during deployment with PEM 9.0.
  • TPA-219 Make pem-agent monitor the bdr_database by default on BDR instances

TPA 23.9

Released: 2022-12-12

Bugfixes

  • TPA-301 Fix auto-detection of cluster_dir for Tower clusters When setting cluster_dir based on the Tower project directory, we now correctly check for the existence of the directory on the controller, and not on the instances being deployed to.
  • TPA-283 Add dependency on psutil, required for Ansible Tower.
  • TPA-278 Remove "umask 0" directive from rsyslog configuration, which previously resulted in the creation of world-readable files such as rsyslogd.pid .
  • TPA-291 Respect the postgres_package_version setting when installing the Postgres server package to obtain pg_receivewal on Barman instances.

TPA 23.8

Released: 2022-11-30

Notable changes

  • TPA-18 Support Ansible Tower 3.8 This release supports execution of deploy.yml (only) on a bare cluster (i.e., with existing servers) through Ansible Tower 3.8. Install TPAexec on the Tower server and run tpaexec setup to create a virtual environment which can be used in Tower Templates to run TPAexec playbooks. Use the --use-ansible-tower and --tower-git-repository configure options to generate a Tower-compatible cluster configuration. For details, see Ansible Tower.

Minor changes

  • TPA-238 Initialise the cluster directory as a git repository If git is available on the system where you run TPAexec, tpaexec configure will now initialise a git repository within the cluster directory by default. If git is not available, it will continue as before. To avoid creating the repository (for example, if you want to store the cluster directory within an existing repository), use the --no-git option.

TPA 23.7

Released: 2022-11-09

Notable changes

  • TPA-234 Support the community release of Ansible 2.9 TPAexec used to require the 2ndQuadrant/ansible fork of Ansible 2.9. In this release, you may instead choose to use the community release of Ansible with the tpaexec setup --use-community-ansible. For now, the default continues to be to use 2ndQuadrant/ansible. This will change in a future release; support for 2ndQuadrant/ansible will be dropped, and Ansible will become the new default.

Minor changes

  • TPA-209 Accept --postgres-version 15 as a valid tpaexec configure option, subsequent to the release of Postgres 15
  • TPA-226 Accept IP addresses in the --hostnames-from file Formerly, the file passed to tpaexec configure was expected to contain one hostname per line. Now it may also contain an optional IP address after each hostname. If present, this address will be set as the ip_address for the corresponding instance in config.yml. (If you specify your own --hostnames-from file, the hostnames will no longer be randomised by default.)
  • TPA-231 Add a new bdr-pre-group-join hook This hook is executed before each node joins the BDR node group. It may be used to change the default replication set configuration that TPAexec provides.
  • TPA-130 Use the postgresql_user module from community.postgresql The updated module from the community.postgresql collection is needed in order to correctly report the task status when using a SCRAM password (the default module always reports changed).
  • TPA-250 Upgrade to the latest versions of various Python dependencies

Bugfixes

  • TPA-220 Ensure LD_LIBRARY_PATH in .bashrc does not start with ":"
  • TPA-82 Avoid removing BDR-internal ${group_name}_ext replication sets
  • TPA-247 Fix "'str object' has no attribute 'node_dsn'" errors on AWS The code no longer assigns hostvars[hostname] to an intermediate variable and expects it to behave like a normal dict later (which works only sometimes). This fixes a regression in 23.6 reported for AWS clusters with PEM enabled, but also fixes other similar errors throughout the codebase.
  • TPA-232 Eliminate a race condition in creating a symlink to generated secrets in the inventory that resulted in "Error while linking: [Errno 17] File exists" errors
  • TPA-252 Restore code to make all BDR nodes publish to the witness-only replication set This code block was inadvertently removed in the v23.6 release as part of the refactoring work done for TPA-193.

TPA 23.6

Released: 2022-09-28

Notable changes

  • TPA-21 Use boto3 (instead of the unmaintained boto2) AWS client library for AWS deployments. This enables SSO login and other useful features.
  • TPA-202 Add harp-config hook. This deploy-time hook executes after HARP is installed and configured and before it is started on all nodes where HARP is installed.

Bugfixes

  • TPA-181 Set default python version to 2 on RHEL 7. Formerly, tpaexec could generate a config.yml with the unsupported combination of RHEL 7 and python 3.
  • TPA-210 Fix aws deployments using existing security groups. Such a deployment used to fail at provision-time but will now work as expected.
  • TPA-189 Remove group_vars directory on deprovision. This fixes a problem that caused a subsequent provision to fail because of a dangling symlink.
  • TPA-175 Correctly configure systemd to leave shared memory segments alone. This only affects source builds.
  • TPA-160 Allow version setting for haproxy and PEM. This fixes a bug whereby latest versions of packages would be installed even if a specific version was specified.
  • TPA-172 Install EFM on the correct set of hosts. EFM should be installed only on postgres servers that are members of the cluster, not servers which have postgres installed for other reasons, such as PEM servers.
  • TPA-113 Serialize PEM agent registration. This avoids a race condition when several hosts try to run pemworker --register-agent at the same time.

TPA 23.5

Released: 2022-08-23

Notable changes

  • TPA-81 Publish tpaexec and tpaexec-deps packages for Ubuntu 22.04 Jammy
  • TPA-26 Support harp-proxy and harp-manager installation on a single node. It is now possible to have both harp-proxy and harp-manager service running on the same target node in a cluster.

TPA 23.4

Released: 2022-08-03

Bugfixes

  • TPA-152 fix an issue with locale detection during first boot of Debian instances in AWS Hosts would fail to complete first boot which would manifest as SSH key negotiation issues and errors with disks not found during deployment. This issue was introduced in 23.3 and is related to TPA-38

TPA 23.3

Released: 2022-08-03

Notable changes

  • TPA-118 Exposed two new options in harp-manager configuration. The first sets HARP harp_db_request_timeout similar to dcs request_timeout but for database connections and the second harp_ssl_password_command specifies a command used to de-obfuscate sslpassword used to decrypt the sslkey in SSL enabled database connection

Minor changes

  • TPA-117 Add documentation update on the use of wildcards in package_version options in tpaexec config.yml. This introduces a warning that unexpected package upgrades can occur during a deploy operation. See documentation in tpaexec-configure.md for more info
  • TPA-38 Add locale files for all versions of Debian, and RHEL 8 and above. Some EDB software, such as Barman, has a requirement to set the user locale to en_US.UTF-8. Some users may wish to also change the locale, character set or language to a local region. This change ensures that OS files provided by libc are installed on AWS instances during firstboot using user-data scripts. The default locale is en_US.UTF-8. See platform_aws.md documentation for more info
  • TPA-23 Add log config for syslog for cluster services Barman, HARP, repmgr, PgBouncer and EFM. The designated log server will store log files received in /var/log/hosts directories for these services
  • TPA-109 Minor refactoring of the code in pgbench role around choosing lock timeout syntax based on a given version of BDR

Bugfixes

  • TPA-147 For clusters that use the source install method some missing packages for Debian and Rocky Linux were observed. Debian receives library headers for krb5 and lz4. On RedHat derived OSs the mandatory packages from the "Development Tools" package group and the libcurl headers have been added
  • TPA-146 Small fix to the method of package selection for clusters installing Postgres 9.6
  • TPA-138 Addresses a warning message on clusters that use the "bare" platform that enable the local-repo configure options. As the OS is not managed by TPAexec in the bare platform we need to inform the user to create the local-repo structure. This previously caused an unhandled error halting the configure progress
  • TPA-135 When using --use-local-repo-only with the "docker" platform and the Rocky Linux image initial removal of existing yum repository configuration on nodes would fail due to the missing commands find and xargs. This change ensures that if the findutils package exists in the source repo it will be installed first
  • TPA-111 Remove a redundant additional argument on the command used to register agents with the PEM server when --enable-pem option is given. Previously, this would have caused no problems as the first argument, the one now removed, would be overridden by the second
  • TPA-108 Restore SELinux file context for postmaster symlink when Postgres is installed from source. Previously, a cluster using a SELinux enabled OS that is installing postgres from source would fail to restart Postgres as the systemd daemon would be unable to read the symlink stored in the Postgres data bin directory. This was discovered in tests using a recently adopted Rocky Linux image in AWS that has SELinux enabled and in enforcing mode by default

TPA 23.2

Released: 2022-07-13

Notable changes

  • Add support for Postgres Backup API for use with Barman and PEM. Accessible through the --enable-pg-backup-api option.
  • SSL certificates can now be created on a per-service basis, for example the server certificate for Postgres Backup API proxy service. Certificates will be placed in /etc/tpa/<service>/<hostname>.cert These certificates can also be signed by a CA certificate generated for the cluster.
  • Placement of Etcd for the BDR-Always-ON architecture When using 'harp_consensus_protocol: etcd', explicitly add 'etcd' to the role for each of the following instances:
    • BDR Primary ('bdr' role)
    • BDR Logical Standby ('bdr' + 'readonly' roles)
    • only for the Bronze layout: BDR Witness ('bdr' + 'witness' roles)
    • only for the Gold layout: Barman ('barman' role) Credit: Gianni Ciolli gianni.ciolli@enterprisedb.com

Minor changes

  • Replace configure argument --2q with --pgextended to reflect product branding changes. Existing configuration will retain expected behaviour.
  • Improve error reporting on Docker platform compatibility checks when using version 18 of docker, which comes with Debian old stable.
  • Add some missing commands to CLI help documentation.
  • Improved error reporting of configure command.
  • Add initial support for building BDR 5 from source. Credit: Florin Irion florin.irion@enterprisedb.com
  • Changes to ensure ongoing compatibility for migration from older versions of Postgres with EDB products.

Bugfixes

  • Fixed an issue which meant packages for etcd were missing when using the download-packages command to populate the local-repo.
  • Fixed an issue affecting the use of efm failover manager and the selection of its package dependencies

TPA 23.1

Released: 2022-06-21

This release requires you to run tpaexec setup after upgrading (and will fail with an error otherwise)

Changes to package installation behavior

In earlier versions, running tpaexec deploy could potentially upgrade installed packages, unless an exact version was explicitly specified (e.g., by setting postgres_package_version). However, this was never a safe, supported, or recommended way to upgrade. In particular, services may not have been safely and correctly restarted after a package upgrade during deploy.

With this release onwards, tpaexec deploy will never upgrade installed packages. The first deploy will install all required packages (either a specific version, if set, or the latest available), and subsequent runs will see that the package is installed, and do nothing further. This is a predictable and safe new default behavior.

If you need to update components, use tpaexec update-postgres. In this release, the command can update Postgres and Postgres-related packages such as BDR or pglogical, as well as certain other components, such as HARP, pgbouncer, and etcd (if applicable to a particular cluster). Future releases will safely support upgrades of more components.

Notable changes

  • Run "harpctl apply" only if the HARP bootstrap config is changed WARNING: This will trigger a single harp service restart on existing clusters when you run tpaexec deploy, because config.yml is changed to ensure that lists are consistently ordered, to avoid unintended changes in future deploys
  • Add tpaexec download-packages command to download all packages required by a cluster into a local-repo directory, so that they can be copied to cluster instances in airgapped/disconnected environments. See air-gapped.md and local-repo.md for details
  • Require --harp-consensus-protocol <etcd|bdr> configure option for new BDR-Always-ON clusters TPAexec no longer supplies a default value here because the choice of consensus protocol can negatively affect failover performance, depending on network latency between data centres/locations, so the user is in a better position to select the protocol most suitable for a given cluster. This affects the configuration of newly-generated clusters, but does not affect existing clusters that use the former default of etcd without setting harp_consensus_protocol explicitly

Minor changes

  • Install openjdk-11 instead of openjdk-8 for EFM on distributions where the older version is not available
  • Accept harp_log_level setting (e.g., under cluster_vars) to override the default harp-manager and harp-proxy log level (info)
  • Configure harp-proxy to use a single multi-host BDR DCS endpoint DSN instead of a list of individual endpoint DSNs, to improve resilience
  • Omit extra connection attributes (e.g., ssl*) from the local (Unix socket) DSN for the BDR DCS for harp-manager

Bugfixes

  • Ensure that harp-manager and harp-proxy are restarted if their config changes
  • Fix harp-proxy errors by granting additional (new) permissions required by the readonly harp_dcs_user
  • Disable BDR4 transaction streaming when CAMO is enabled If bdr.enable_camo is set, we must disable bdr.default_streaming_mode, which is not compatible with CAMO-protected transactions in BDR4. This will cause a server restart on CAMO-enabled BDR4 clusters (which could not work with streaming enabled anyway).
← Prev

Trusted Postgres Architect 23.12 release notes