Patroni cluster management commands v23.41.0

Patroni can be used as a single master failover manager with the M1 architecture using the following command options.

tpaexec configure cluster_name -a M1 --enable-patroni --postgresql 14

Or by setting the config.yml option

cluster_vars:
  failover_manager: patroni

TPA is able to deploy Patroni clusters using either patroni packages (from PGDG repositories) or edb-patroni packages (from EDB repositories). You can configure that through the patroni_package_flavour option under cluster_vars in the config.yml, which can also be set through the --patroni-package-flavour command-line argument. If no patroni_package_flavour is explicitly set, TPA will attempt to infere the flavour based on the configured repositories: if EDB repositories were configured, implicitly select edb flavour, otherwise implicitly select community flavour.

TPA configure will add 3 etcd nodes, and may add 2 haproxy nodes if you specify the option --enable-haproxy. Etcd is used for the Distributed Configuration Store (DCS). Patroni supports other DCS backends, but they are not currently supported by EDB or TPA.

As an alternative to HAProxy, you can use the --enable-pgbouncer option to configure PgBouncer in the Postgres nodes. PgBouncer will be configured to pool connections for the primary. Patroni will be configured to reconfigure PgBouncer upon failovers or switchovers in the cluster, so PgBouncer follows the new primary Postgres instance.

TPA uses Patroni's feature of converting an existing PostgreSQL cluster. This allows for TPA to initialise and manage configuration. Once the PostgreSQL cluster has been created, Patroni will take the management over. TPA will then remove any postgres configuration files used during setup.

Once set up, Postgres can continue to be managed using TPA and settings in config.yml for the cluster. You can also use Patroni interfaces, such as the command line patronictl and the REST API, but it is recommended to use TPA methods wherever possible.

Patroni package version

By default, TPA installs the latest available version of Patroni.

The version of the Patroni package that is installed can be specified by including patroni_package_version: xxx under the cluster_vars section of the config.yml file.

cluster_vars:
    …
    patroni_package_version: '4.0.0-1PGDG.rhel8`'
    …

You may use any version specifier that apt or yum would accept.

If your version does not match, try appending a * wildcard. This is often necessary when the package version has an epoch qualifier like 2:....

TPA provides robust options for securing the Patroni REST API. This is controlled primarily by the patroni_ssl_enabled and patroni_authentication_mode variables.

TLS Encryption

Setting patroni_ssl_enabled: true enables TLS encryption for the REST API. TPA will automatically:

• Generate a TPA cluster CA and server certificates for each Patroni node.
• Configure the Patroni REST API to serve traffic over HTTPS.
• Configure clients (like patronictl and HAProxy) to connect via HTTPS and validate the server certificate using the CA.

Client Authentication

Once TLS encryption is enabled (patroni_ssl_enabled: true), you can choose an authentication mode using the patroni_authentication_mode variable:

• basic (default): Requires clients to provide the configured Patroni REST API username and password. TPA generates these credentials automatically.

• mtls: Requires clients to present a valid TLS certificate signed by the trusted cluster CA. TPA automatically:

  • Configures Patroni REST API to require and validate client certificates (verify_client: required).
  • Configures clients (patronictl, HAProxy) with the necessary client certificates and keys.

API Access Control (Allowlist)

For enhanced security, Patroni's REST API is automatically configured with an allowlist. This list restricts access to potentially harmful API endpoints (those using POST, PUT, PATCH, DELETE) to only the other Patroni nodes within the cluster, based on the hostnames defined in the TPA inventory. Read-only endpoints (like health checks) remain accessible from other sources (e.g., HAProxy).

These configuration variables can be used to control certain behaviours in the deployment of Patroni in TPA.

tpaexec store-password <your_custom_username> .

Patroni configuration file settings

Configuration for patroni is built from three layers, starting with defaults set by the Patroni daemon, config loaded from the DCS, and finally from local configuration. The last can be controlled from either configuration file and overrides via the environment. TPA controls the configuration file and values are built up in this order.

DCS config to be sent to the API and stored in the bootstrap section of the config file:

• TPA vars for postgres are loaded into the DCS settings, see postgresql.conf.md. Some features are not supported, see notes below.
• Patroni defaults for DCS settings
• User supplied defaults in patroni_dynamic_conf_settings, if you want to override any DCS settings you can do that here.

Local config stored in the YAML configuration file:

• bootstrap.dcs loaded from previous steps above.
• configuration enabled by feature flags, such as patroni_ssl_enabled, see table above.
• then finally overloaded from user supplied settings, the patroni_conf_settings option. If you want to change or add configuration not controlled by a feature flag then this is the best place to do it.

Please note that configuration is merged on top of configuration generated by TPA from cluster information, such as IP addresses, port numbers, cluster roles, etc. Exercise caution in what you override as this might affect the stable operation of the cluster.

As Patroni stores all postgres configuration in the DCS and controls how and when this is distributed to postgres, some features of TPA are incompatible with patroni:

• It is not possible to change the template used to generate postgresql.conf with the setting postgres_conf_template.
• You cannot change the location of Postgres config files with the setting postgres_conf_dir.

Patroni configuration in TPA config.yml

You can override single values:

cluster_vars:
  patroni_conf_settings:
    bootstrap:
      dcs:
        ttl: 120

Or full blocks (with an example from Patroni docs):

cluster_vars:
  patroni_conf_settings:
    restapi:
      http_extra_headers:
        'X-Frame-Options': 'SAMEORIGIN'
        'X-XSS-Protection': '1; mode=block'
        'X-Content-Type-Options': 'nosniff'
      https_extra_headers:
        'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'

If you want to negate a value or section that is present in the default TPA config vars you can set the value to null. This will cause patroni to ignore this section when loading the config file.

For example the default TPA config for log is

log:
  dir: /var/log/patroni

To turn off logging add this to config.yml:

cluster_vars:
  patroni_conf_settings:
    log: null

Etcd Integration

When etcd is used as the DCS, TPA automatically configures Patroni to connect to it based on the etcd security configuration:

• Protocol: Uses http or https based on etcd_ssl_enabled.
• CA Certificate: Provides the cluster CA certificate path when TLS is enabled.
• Authentication:
  • If etcd_authentication_mode is basic, Patroni is configured with the username (patroni_etcd_user) and its generated password.
  • If etcd_authentication_mode is mtls, Patroni is configured with the paths to its client certificate and key.

TPA provides these minimal set of tools for managing Patroni clusters.

Status

To see the current status of the TPA cluster according to Patroni run

tpaexec status cluster_name

Switchover

To perform a switchover to a replica node (e.g. to perform maintenance) run the command

tpaexec switchover cluster_name new_primary

The new_primary argument must be the name of an existing cluster node that is currently running as a healthy replica. Checks will be performed to ensure this is true before a switchover is performed.

Once a switchover has been performed it is recommended that you run deploy and test to ensure a healthy cluster.

tpaexec deploy cluster_name
tpaexec test cluster_name

TPA will detect the current role of nodes during deploy regardless of what config.yml contains, for example if a different node is the leader.

Minor update for patroni using tpaexec upgrade

When trying to upgrade to a specific package version, ensure the patroni_package_version in config.yml is updated to reflect the desired version.

The desired version can also be passed as an extra argument to the tpaexec upgrade command with:

tpaexec upgrade <cluster_dir> -e patroni_package_version="<desired version>" --components=patroni

Refer to the section on package version selection and upgrade for more information.

To select Patroni for upgrade, ensure the --components flag passed to the tpaexec upgrade command contains patroni (or all)

Refer to the section on component selection for upgrade for more information.