High Availability for EDB Postgres Advanced Server Using EFM and PgPool: A Complete Setup Guide

This guide walks through a complete High Availability (HA) setup for EDB Postgres Advanced Server (EPAS) using EDB Failover Manager (EFM) and PgPool-II. Together, these two components deliver a robust, two-layer HA architecture: EFM monitors the PostgreSQL primary-standby pair and automatically promotes the standby when the primary fails, while PgPool sits in front of the database nodes to provide connection pooling, read/write load balancing, and watchdog-based HA across three PgPool instances behind a single Delegate (Virtual) IP.

Tested versions: EPAS v16.4.1, EFM v4.9, PgPool v4.4.5 on RHEL v8. The Environment Details section lists the latest available versions of EFM and PgPool. If you are running those newer versions, see Section 7: Key Parameter Changes for the configuration differences you need to account for before applying this runbook.

Environment Details

• Database: EDB Postgres Advanced Server (EPAS) v16.4.1
• EFM (Latest): EDB Failover Manager v5.3 (this guide was tested with v4.9)
• PgPool (Latest): edb-pgpool v4.7.1 (this guide was tested with v4.4.5)
• OS: RHEL v8

Node Layout

Section 1: Setting Up the EDB Repository

Run the following command on all six nodes (three EFM nodes and three PgPool nodes) to configure the EDB software repository. Replace <TOKEN> With your EDB subscription token.

root# curl -1sLf 'https://downloads.enterprisedb.com/<TOKEN>/enterprise/setup.rpm.sh' | sudo -E bash
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.
Executing the setup script for the 'enterprisedb/enterprise' repository ...
   OK: Updating the dnf cache to fetch the new repository metadata ...
   OK: The repository has been installed successfully - You're ready to rock!

Note: Replace <TOKEN> With your own EDB subscription token from https://www.enterprisedb.com/repos.

Section 2: Installing Packages

Step 1: Install Packages on EFM Nodes (Primary, Standby, and Witness)

Run the following on all three EFM nodes. This installs EPAS v16, the EFM agent, and the PgPool extensions that EFM uses to call PCP commands when detaching/attaching nodes from/to the PgPool load balancer.

root# sudo dnf -y install edb-as16-server edb-efm49 edb-as16-pgpool44-extensions edb-pgpool44
Updating Subscription Management repositories.
:
:
Installed:
  edb-as16-pgpool44-extensions-4.4.5-2.el8.x86_64    edb-as16-server-16.4.1-1.el8.x86_64
  edb-as16-server-client-16.4.1-1.el8.x86_64          edb-as16-server-contrib-16.4.1-1.el8.x86_64
  edb-as16-server-core-16.4.1-1.el8.x86_64            edb-efm49-4.9-1.el8.x86_64
  edb-pgpool44-4.4.5-2.el8.x86_64                     edb-pgpool44-libs-4.4.5-2.el8.x86_64
Complete!

Note: For the latest EFM version (5.3), replace edb-efm49 with edb-efm53 and edb-as16-pgpool44-extensions / edb-pgpool44 with edb-as16-pgpool47-extensions / edb-pgpool47.

Step 2: Install Packages on PgPool Nodes (All 3 Nodes)

On the three dedicated PgPool nodes, install only the PgPool packages and the EPAS PgPool extensions. A full EPAS server installation is not required on these nodes.

root# sudo dnf -y install edb-as16-pgpool44-extensions edb-pgpool44
:
:
Installed:
  edb-as16-pgpool44-extensions-4.4.5-2.el8.x86_64    edb-pgpool44-4.4.5-2.el8.x86_64
  edb-pgpool44-libs-4.4.5-2.el8.x86_64
Complete!

Section 3: PostgreSQL Database Configuration

On the Primary Node (192.168.40.179)

Step 1: Initialize the database

Switch to the enterprisedb OS user and initialize a new EPAS data directory.

root# su - enterprisedb
enterprisedb$ cd /usr/edb/as16/bin/
enterprisedb$ ./initdb -D /var/lib/edb/as16/data/
The files belonging to this database system will be owned by user "enterprisedb".
:
Success. You can now start the database server using:
    pg_ctl -D /var/lib/edb/as16/data/ -l logfile start

Step 2: Create the WAL archive directory

EFM uses WAL archiving during standby rebuild. Create a local archive directory owned by the enterprisedb user.

root# mkdir /tmp/archivedir/
root# chown enterprisedb:enterprisedb /tmp/archivedir/
root# chmod 777 /tmp/archivedir/

Step 3: Configure postgresql.conf for streaming replication

Edit /var/lib/edb/as16/data/postgresql.conf And set the following parameters. wal_level = replica enables streaming replication; archive_mode and archive_command enable WAL file archiving for standby recovery; max_wal_senders limits the number of concurrent replication connections; wal_keep_size retains recent WAL segments on the primary to reduce the chance of a standby falling too far behind; password_encryption = md5 is required because PgPool's pool_passwd file uses MD5-hashed passwords.

archive_mode = on
archive_command = 'cp %p /tmp/archivedir/%f'
wal_level = replica
max_wal_senders = 10
wal_keep_size = 100
password_encryption = md5

Step 4: Configure pg_hba.conf

Edit /var/lib/edb/as16/data/pg_hba.conf And add entries for streaming replication, EFM agent connections, and PgPool node connections. Apply the same pg_hba.conf on both the Primary and Standby nodes.

# Streaming Replication
host    replication     all             192.168.40.182/32       trust   # Standby
host    replication     all             192.168.40.179/32       trust   # Primary

# EFM agent access
host    all     all             192.168.40.179/32               trust   # Primary
host    all     all             192.168.40.182/32               trust   # Standby
host    all     all             192.168.40.183/32               trust   # Witness

# PgPool node access
host    all     all             192.168.40.184/32               md5     # PgPool Node 1
host    all     all             192.168.40.180/32               md5     # PgPool Node 2
host    all     all             192.168.40.167/32               md5     # PgPool Node 3
host    all     all             192.168.40.1/32                 md5     # PgPool Delegate IP

Step 5: Start the primary database

root# systemctl start edb-as-16.service

Step 6: Set a password for the enterprisedb superuser

EFM encrypts this password and stores it in efm.properties. PgPool also uses it for health checks and SR checks.

edb=# ALTER USER enterprisedb WITH PASSWORD 'edb';
ALTER ROLE

Note: Use a strong password in production. Replace 'edb' with your actual password throughout this guide.

On the Standby Node (192.168.40.182)

Step 7: Take a base backup from the primary

pg_basebackup copies the primary data directory to the standby and writes the standby.signal file (via -R) so EPAS starts up in standby/recovery mode automatically.

enterprisedb$ /usr/edb/as16/bin/pg_basebackup -h 192.168.40.179 \
  -U enterprisedb -Fp -D /var/lib/edb/as16/data/ -R

Step 8: Configure recovery parameters in postgresql.conf

After the base backup completes, add the following two parameters to /var/lib/edb/as16/data/postgresql.conf on the standby. restore_command tells the standby how to fetch archived WAL segments from the primary if the WAL stream lags; recovery_target_timeline = 'latest' ensures the standby follows the latest timeline after a failover or switchover.

restore_command = 'scp -r enterprisedb@192.168.40.179:/tmp/archivedir/%f %p'
recovery_target_timeline = 'latest'

Step 9: Start the standby database

root# systemctl start edb-as-16.service

Step 10: Verify streaming replication on the primary

Connect to the primary and confirm the standby is connected and streaming in async mode.

edb=# SELECT pid, usename, client_addr, sync_state, reply_time FROM pg_stat_replication;
-[ RECORD 1 ]----+---------------------------------
pid              | 37430
usesysid         | 10
usename          | enterprisedb
client_addr      | 192.168.40.182
sync_state       | async
reply_time       | 09-SEP-24 16:09:13.956699 +05:30

Section 4: PgPool Configuration

PgPool is configured on all three PgPool nodes. Most parameters are identical across nodes — the exceptions are wd_priority (must be unique per node), and the pgpool_node_id file (0, 1, or 2).

Step 1: Create configuration files from samples (all 3 PgPool nodes)

Copy the sample files to create the live config files and set enterprisedb Ownership so PgPool can read them.

root# ls -l /etc/sysconfig/edb/pgpool4.4/
-rw-r--r--. 1 root         root           858 May 16 16:59 pcp.conf.sample
-rw-------. 1 root         root         52774 May 16 16:59 pgpool.conf.sample
-rw-r--r--. 1 root         root          1183 May 16 16:59 pgpool_remote_start.sample
-rw-r--r--. 1 root         root          3476 May 16 16:59 pool_hba.conf.sample

root# cd /etc/sysconfig/edb/pgpool4.4/
root# cp pgpool.conf.sample pgpool.conf
root# cp pcp.conf.sample pcp.conf
root# cp pool_hba.conf.sample pool_hba.conf
root# chown enterprisedb:enterprisedb pgpool.conf pcp.conf pool_hba.conf

Step 2: Key pgpool.conf parameters (common template)

The following parameters must be present in pgpool.conf on every PgPool node. The backend, watchdog, and heartbeat IP addresses are the same on all three nodes. Only wd_priority differs (set to 1, 2, or 3 per node).

## Clustering mode and listen settings — same on all nodes
backend_clustering_mode = 'streaming_replication'
listen_addresses = '*'
port = 9999
pcp_listen_addresses = '*'
pcp_port = 9898

## Backend (DB) nodes — same on all nodes
backend_hostname0 = '192.168.40.179'    # Primary DB
backend_port0 = 5444
backend_weight0 = 1
backend_data_directory0 = '/var/lib/edb/as16/data'
backend_flag0 = 'ALLOW_TO_FAILOVER'
backend_application_name0 = 'node1'

backend_hostname1 = '192.168.40.182'    # Standby DB
backend_port1 = 5444
backend_weight1 = 1
backend_data_directory1 = '/var/lib/edb/as16/data'
backend_flag1 = 'ALLOW_TO_FAILOVER'
backend_application_name1 = 'node2'

## Load balancing and authentication
enable_pool_hba = on
pool_passwd = 'pool_passwd'
load_balance_mode = on

## SR check and health check — same on all nodes
sr_check_user = 'enterprisedb'
sr_check_password = 'edb'
sr_check_database = 'edb'
health_check_user = 'enterprisedb'
health_check_password = 'edb'
health_check_database = 'edb'

## Additional recommended settings
child_life_time = 0
sr_check_period = 10
search_primary_node_timeout = 10

## Watchdog settings — same on all nodes
use_watchdog = on
hostname0 = '192.168.40.180'
wd_port0 = 9000
pgpool_port0 = 9999
hostname1 = '192.168.40.184'
wd_port1 = 9000
pgpool_port1 = 9999
hostname2 = '192.168.40.167'
wd_port2 = 9000
pgpool_port2 = 9999

## Virtual IP and interface commands — same on all nodes
delegate_ip = '192.168.40.1'
if_cmd_path = '/usr/sbin'
if_up_cmd = '/usr/bin/sudo /sbin/ip addr add $_IP_$/24 dev ens160'
if_down_cmd = '/usr/bin/sudo /sbin/ip addr del $_IP_$/24 dev ens160'
arping_cmd = '/usr/bin/sudo /usr/sbin/arping -U $_IP_$ -w 1 -I ens160'

## Heartbeat — same on all nodes
wd_lifecheck_method = 'heartbeat'
wd_interval = 10
wd_life_point = 3
heartbeat_hostname0 = '192.168.40.180'
heartbeat_port0 = 9694
heartbeat_device0 = 'ens160'
heartbeat_hostname1 = '192.168.40.184'
heartbeat_port1 = 9694
heartbeat_device1 = 'ens160'
heartbeat_hostname2 = '192.168.40.167'
heartbeat_port2 = 9694
heartbeat_device2 = 'ens160'

## wd_priority — UNIQUE PER NODE
wd_priority = 1    # Node 1: set to 1; Node 2: set to 2; Node 3: set to 3

Note: if_up_cmd, if_down_cmd, and arping_cmd contain the network interface name (ens160) and subnet mask (/24). Verify these match your actual interface name and network configuration. If PgPool nodes use a different interface name, adjust accordingly per node.

Step 3: Exact pgpool.conf per node

Only wd_priority changes across nodes:

• PgPool Node 1 (192.168.40.167): wd_priority = 3  This node becomes the watchdog Leader because it has the highest priority.
• PgPool Node 2 (192.168.40.180): wd_priority = 1
• PgPool Node 3 (192.168.40.184): wd_priority = 2

All other parameters in the template above remain the same on every node.

Step 4: Set the pgpool_node_id file (all 3 PgPool nodes)

Each PgPool node needs a unique numeric ID file. Create and set ownership on each node:

PgPool Node 1 (192.168.40.167):

root# echo 0 > /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# chown enterprisedb:enterprisedb /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# cat /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
0

PgPool Node 2 (192.168.40.180):

root# echo 1 > /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# chown enterprisedb:enterprisedb /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# cat /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
1

PgPool Node 3 (192.168.40.184):

root# echo 2 > /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# chown enterprisedb:enterprisedb /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
root# cat /etc/sysconfig/edb/pgpool4.4/pgpool_node_id
2

Step 5: Set up pcp.conf (all 3 PgPool nodes)

pcp.conf Stores the username and MD5-hashed password used to authenticate PCP management connections. EFM uses PCP to detach/attach nodes during failover. Generate the MD5 hash for the enterprisedb user password using the pg_md5 utility:

root# /usr/edb/pgpool4.4/bin/pg_md5 --username enterprisedb -p
password: <enter your password>
<MD5_HASH_OUTPUT>

Add the result to pcp.conf in the format username:md5hash:

root# cat /etc/sysconfig/edb/pgpool4.4/pcp.conf
# USERID:MD5PASSWD
enterprisedb:<MD5_HASH_OUTPUT>

Note: The hash above is the MD5 of your database password. Generate it using pg_md5 --username enterprisedb -p on each node and use the output. Never use plain-text passwords in pcp.conf.

Step 6: Set up pool_passwd (all 3 PgPool nodes)

pool_passwd maps database usernames to passwords for client authentication when enable_pool_hba = on. The format is username:password (plain text or MD5). The username must match the PostgreSQL superuser exactly.

root# cat /etc/sysconfig/edb/pgpool4.4/pool_passwd
enterprisedb:edb

Note: In production, use MD5-hashed passwords in pool_passwd. Generate with: /usr/edb/pgpool4.4/bin/pg_md5 -m -u enterprisedb <password>. This updates pool_passwd in place with a secure hash.

Step 7: Set up pool_hba.conf (all 3 PgPool nodes)

pool_hba.conf Controls which clients are allowed to connect through PgPool and which authentication method to use. It mirrors the role of pg_hba.conf but at the PgPool layer.

root# cat /etc/sysconfig/edb/pgpool4.4/pool_hba.conf
host    all     all     192.168.40.179/32     md5   # Primary database IP
host    all     all     192.168.40.182/32     md5   # Standby database IP
host    all     all     192.168.40.0/24       md5   # PgPool delegate IP subnet

Step 8: Grant sudo privileges for VIP management (all 3 PgPool nodes)

PgPool needs to run ip addr add/del and arping as root to manage the Delegate IP. Grant the enterprisedb user passwordless sudo:

root# vi /etc/sudoers
## Allow root to run any commands anywhere
root            ALL=(ALL)       ALL
enterprisedb    ALL=(ALL)       NOPASSWD: ALL

Step 9: Start PgPool on all 3 nodes

root# systemctl start edb-pgpool-4.4.service
root# systemctl status edb-pgpool-4.4.service
* edb-pgpool-4.4.service - pgpool-II service script for EDB Postgres Advanced Server
   Loaded: loaded (/usr/lib/systemd/system/edb-pgpool-4.4.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2024-09-09 22:01:56 IST; 3s ago

Step 10: Verify watchdog cluster health

Run pcp_watchdog_info from any PgPool node. The output should show all 3 nodes as MEMBER, one node as LEADER, and the cluster quorum as QUORUM EXIST.

root# /usr/edb/pgpool4.4/bin/pcp_watchdog_info -p 9898 -U enterprisedb -v
Password:
Watchdog Cluster Information
Total Nodes              : 3
Remote Nodes             : 2
Member Remote Nodes      : 2
Alive Remote Nodes       : 2
Nodes required for quorum: 2
Quorum state             : QUORUM EXIST
Local node escalation    : YES
Leader Node Name         : 192.168.40.167:9999 Linux localhost.localdomain
Leader Host Name         : 192.168.40.167

Watchdog Node Information
Node Name         : 192.168.40.167:9999 Linux localhost.localdomain
Host Name         : 192.168.40.167
Delegate IP       : 192.168.40.1
Pgpool port       : 9999
Watchdog port     : 9000
Node priority     : 3
Status            : 4
Status Name       : LEADER
Membership Status : MEMBER

Node Name         : 192.168.40.180:9999 Linux localhost.localdomain
Host Name         : 192.168.40.180
Delegate IP       : 192.168.40.1
Pgpool port       : 9999
Watchdog port     : 9000
Node priority     : 1
Status            : 7
Status Name       : STANDBY
Membership Status : MEMBER

Node Name         : 192.168.40.184:9999 Linux localhost.localdomain
Host Name         : 192.168.40.184
Delegate IP       : 192.168.40.1
Pgpool port       : 9999
Watchdog port     : 9000
Node priority     : 2
Status            : 7
Status Name       : STANDBY
Membership Status : MEMBER

Step 11: Verify Delegate IP assignment on the Leader node (192.168.40.167)

root# hostname -I
192.168.40.167 192.168.40.1

The Delegate IP 192.168.40.1 Is visible on the Leader node, confirming the watchdog has assigned it correctly. Clients should connect to this IP on port 9999.

Section 5: EFM Configuration

Overview

EFM is configured without its own VIP (the VIP is managed by PgPool watchdog). The key integration point is the pgpool.enable=true setting combined with script.load.balancer.attach and script.load.balancer.detach — these tell EFM to call the efm_pgpool_functions Script to attach or detach a database node from PgPool load balancing whenever a failover or switchover occurs.

• The .pcppass file must exist at /etc/edb/efm-4.9/, owned by the efm user with 600 permissions.
• PgPool packages (edb-pgpool44) must be installed on EFM nodes so the efm_pgpool_functions script and PgPool binaries are available.
• EFM nodes must be able to reach PgPool nodes on the PCP port (9898), the PgPool port (9999), and the Watchdog port (9000).

Step 1: Create the .pcppass file on all EFM nodes

The .pcppass File stores PCP credentials so EFM can issue PCP commands to the PgPool cluster without interactive password prompts. The format is <delegate_ip>:<pcp_port>:<pcp_user>:<pcp_password>.

root# vi /etc/edb/efm-4.9/.pcppass
192.168.40.1:9898:enterprisedb:edb

root# chown efm:efm /etc/edb/efm-4.9/.pcppass
root# chmod 600 /etc/edb/efm-4.9/.pcppass

root# ls -lrth /etc/edb/efm-4.9/.pcppass
-rw-------. 1 efm efm 37 Sep  6 20:37 /etc/edb/efm-4.9/.pcppass

Step 2: Create efm.properties and efm.nodes from template files (all 3 EFM nodes)

root# cd /etc/edb/efm-4.9/
root# ls -l
total 32
-rw-r--r--. 1 root root   139 May  2 23:08 efm.nodes.in
-rw-r--r--. 1 root root 27845 May  2 23:08 efm.properties.in

root# cp efm.nodes.in efm.nodes
root# cp efm.properties.in efm.properties
root# chown efm:efm efm.properties efm.nodes

root# ls -l /etc/edb/efm-4.9/
-rw-r--r--. 1 efm  efm    139 Sep  9 21:26 efm.nodes
-rw-r--r--. 1 root root   139 May  2 23:08 efm.nodes.in
-rw-r--r--. 1 efm  efm  27845 Sep  9 21:26 efm.properties
-rw-r--r--. 1 root root 27845 May  2 23:08 efm.properties.in

Step 3: Populate efm.nodes with all three EFM node addresses

The efm.nodes file lists the IP and EFM port (default: 7800) of every node in the cluster. This is the same content on all three nodes.

root# cat /etc/edb/efm-4.9/efm.nodes
# List of node address:port combinations separated by whitespace.
# The list should include at least the membership coordinator's address.
192.168.40.179:7800
192.168.40.182:7800
192.168.40.183:7800

Step 4: Configure efm.properties on each node

Most parameters are identical across all three nodes. The per-node differences are: bind.address (set to the node's own IP) and is.witness (true only on the Witness node).

Note: To generate the encrypted password for db.password.encrypted, run: /usr/edb/efm-4.9/bin/efm encrypt efm And enter the database user's password when prompted.

Primary Node (192.168.40.179)

db.user=enterprisedb
db.password.encrypted=YOUR_ENCRYPTED_DB_PASSWORD
db.port=5444
db.database=edb
db.service.owner=enterprisedb
db.service.name=edb-as-16.service
db.bin=/usr/edb/as16/bin
db.data.dir=/var/lib/edb/as16/data/
user.email=your-email@example.com
from.email=efm@localhost
bind.address=192.168.40.179:7800
is.witness=false
enable.stop.cluster=true
stop.isolated.primary=true
stop.failed.primary=true
primary.shutdown.as.failure=true
auto.failover=true
auto.resume.period=10
pgpool.enable=true
pcp.user=enterprisedb
pcp.host=192.168.40.1
pcp.port=9898
pcp.pass.file=/etc/edb/efm-4.9/.pcppass
pgpool.bin=/usr/edb/pgpool4.4/bin
script.load.balancer.attach=/usr/edb/efm-4.9/bin/efm_pgpool_functions attach efm %h %t
script.load.balancer.detach=/usr/edb/efm-4.9/bin/efm_pgpool_functions detach efm %h %t

Standby Node (192.168.40.182)

db.user=enterprisedb
db.password.encrypted=YOUR_ENCRYPTED_DB_PASSWORD
db.port=5444
db.database=edb
db.service.owner=enterprisedb
db.service.name=edb-as-16.service
db.bin=/usr/edb/as16/bin
db.data.dir=/var/lib/edb/as16/data/
user.email=your-email@example.com
from.email=efm@localhost
bind.address=192.168.40.182:7800
is.witness=false
enable.stop.cluster=true
stop.isolated.primary=true
stop.failed.primary=true
primary.shutdown.as.failure=true
auto.failover=true
auto.resume.period=10
pgpool.enable=true
pcp.user=enterprisedb
pcp.host=192.168.40.1
pcp.port=9898
pcp.pass.file=/etc/edb/efm-4.9/.pcppass
pgpool.bin=/usr/edb/pgpool4.4/bin
script.load.balancer.attach=/usr/edb/efm-4.9/bin/efm_pgpool_functions attach efm %h %t
script.load.balancer.detach=/usr/edb/efm-4.9/bin/efm_pgpool_functions detach efm %h %t

Witness Node (192.168.40.183)

db.user=enterprisedb
db.password.encrypted=YOUR_ENCRYPTED_DB_PASSWORD
db.port=5444
db.database=edb
db.service.owner=enterprisedb
db.service.name=edb-as-16.service
db.bin=/usr/edb/as16/bin
db.data.dir=/var/lib/edb/as16/data/
user.email=your-email@example.com
from.email=efm@localhost
bind.address=192.168.40.183:7800
is.witness=true
enable.stop.cluster=true
stable.nodes.file=true
auto.failover=true
auto.reconfigure=true
promotable=true
auto.resume.period=10
pgpool.enable=true
pcp.user=enterprisedb
pcp.host=192.168.40.1
pcp.port=9898
pcp.pass.file=/etc/edb/efm-4.9/.pcppass
pgpool.bin=/usr/edb/pgpool4.4/bin
script.load.balancer.attach=/usr/edb/efm-4.9/bin/efm_pgpool_functions attach efm %h %t
script.load.balancer.detach=/usr/edb/efm-4.9/bin/efm_pgpool_functions detach efm %h %t

Key parameters explained:

• db.password.encrypted: The EFM-encrypted form of the database password. Generate with /usr/edb/efm-4.9/bin/efm encrypt efm.
• stop.isolated.primary: If the primary can no longer reach any standby or witness, EFM stops it to prevent split-brain.
• stop.failed.primary: After a failover, EFM stops the old primary's database service if it is still running.
• primary.shutdown.as.failure: Treats a clean primary shutdown as a failure, triggering automatic promotion of the standby.
• auto.resume.period: Seconds EFM waits before attempting to re-add a failed agent to the cluster. (In EFM 5.0+, this parameter is split see Section 7.)
• pgpool.enable: Enables the EFM-PgPool integration. EFM will call efm_pgpool_functions attach/detach scripts on role changes.
• pcp.host: Points to the Delegate (Virtual) IP so PCP commands always reach the current PgPool watchdog leader.
• is.witness=true: (Witness only) This node participates in quorum decisions but does not run a database.
• stable.nodes.file=true: (Witness only) EFM will update efm.nodes to only include stable (currently reachable) nodes.

Step 5: Start the EFM service on all nodes

Start EFM in this order: Primary -> Standby -> Witness.

root# systemctl start edb-efm-4.9.service

Step 6: Verify EFM service status

root# systemctl status edb-efm-4.9.service
* edb-efm-4.9.service - EnterpriseDB Failover Manager 4.9
   Loaded: loaded (/usr/lib/systemd/system/edb-efm-4.9.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2024-09-10 18:58:46 IST; 13min ago
  Process: 62367 ExecStart=/bin/bash -c /usr/edb/efm-4.9/bin/runefm.sh start ${CLUSTER}

Step 7: Verify the EFM cluster status

Run efm cluster-status efm from any EFM node. The expected output shows Primary UP, Standby UP, and Witness N/A. The "Standby database(s) in sync with primary. It is safe to promote." message confirms the cluster is ready for a failover or switchover.

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address              DB       VIP
    ----------------------------------------------------------------
    Primary     192.168.40.179       UP
    Standby     192.168.40.182       UP
    Witness     192.168.40.183       N/A

Allowed node host list:
    192.168.40.179 192.168.40.182 192.168.40.183
Membership coordinator: 192.168.40.179
Standby priority host list:
    192.168.40.182

Promote Status:
    DB Type     Address              WAL Received LSN   WAL Replayed LSN   Info
    ---------------------------------------------------------------------------
    Primary     192.168.40.179                          0/70001C0
    Standby     192.168.40.182       0/7000000          0/70001C0

Standby database(s) in sync with primary. It is safe to promote.

Section 6: Verification Commands

Once both PgPool and EFM are running, use these commands to verify the full stack health. Connect through the Delegate IP (port 9999) to exercise PgPool's load balancing.

show pool_nodes — view DB backend status and current primary/standby roles as seen by PgPool:

enterprisedb$ /usr/edb/as16/bin/psql -p 9999 -h 192.168.40.1 -d edb -U enterprisedb \
  -c "show pool_nodes"

 node_id |    hostname    | port | status | pg_status | lb_weight |  role   | pg_role | select_cnt | load_balance_node | replication_delay
---------+----------------+------+--------+-----------+-----------+---------+---------+------------+-------------------+------------------
 0       | 192.168.40.179 | 5444 | up     | unknown   | 0.500000  | primary | unknown | 0          | false             | 0
 1       | 192.168.40.182 | 5444 | up     | unknown   | 0.500000  | standby | unknown | 0          | true              | 0

SHOW POOL_BACKEND_STATS — view per-backend query statistics:

enterprisedb$ /usr/edb/as16/bin/psql -p 9999 -h 192.168.40.1 -d edb -U enterprisedb \
  -c "SHOW POOL_BACKEND_STATS"

 node_id |    hostname    | port | status |  role   | select_cnt | insert_cnt | update_cnt | delete_cnt | ddl_cnt | other_cnt | panic_cnt | fatal_cnt | error_cnt
---------+----------------+------+--------+---------+------------+------------+------------+------------+---------+-----------+-----------+-----------+-----------
 0       | 192.168.40.179 | 5444 | up     | primary | 0          | 0          | 0          | 0          | 0       | 4         | 0         | 0         | 0
 1       | 192.168.40.182 | 5444 | up     | standby | 0          | 0          | 0          | 0          | 0       | 1         | 0         | 0         | 0

SHOW POOL_HEALTH_CHECK_STATS — view health check history per backend node:

enterprisedb$ /usr/edb/as16/bin/psql -p 9999 -h 192.168.40.1 -d edb -U enterprisedb \
  -c "SHOW POOL_HEALTH_CHECK_STATS"

 node_id |    hostname    | port | status |  role   | last_status_change  | total_count | success_count | fail_count
---------+----------------+------+--------+---------+---------------------+-------------+---------------+------------
 0       | 192.168.40.179 | 5444 | up     | primary | 2024-09-10 20:59:44 | 0           | 0             | 0
 1       | 192.168.40.182 | 5444 | up     | standby | 2024-09-10 20:59:44 | 0           | 0             | 0

Section 7: Key Parameter Changes: EFM 5.x and PgPool 4.5+

This guide was tested with EFM v4.9 and PgPool v4.4.5. If you are deploying with the latest versions (EFM v5.3, PgPool v4.7.1), the following parameter and packaging changes apply.

EFM: Changes from v4.9 to v5.3

The most impactful change is the removal of auto.resume.period in EFM 5.0. This single parameter was replaced by two new parameters that give you separate control over auto-resume behaviour on startup vs. after a database failure.

Package and path changes for EFM 5.3:

• Package name: edb-efm49 -> edb-efm53
• Config directory: /etc/edb/efm-4.9/ -> /etc/edb/efm-5.3/
• Binary directory: /usr/edb/efm-4.9/bin/ -> /usr/edb/efm-5.3/bin/
• Systemd service: edb-efm-4.9.service -> edb-efm-5.3.service

PgPool: Changes from v4.4.5 to v4.7.1

The most impactful change in PgPool 4.6 is that several critical parameters now default to an empty string instead of 'nobody'. If these parameters are left blank, PgPool may fail to perform SR checks, health checks, or watchdog lifecheck correctly. Always set them explicitly.

Package and path changes for PgPool 4.7.1:

• Package names: edb-pgpool44 -> edb-pgpool47; edb-as16-pgpool44-extensions -> edb-as16-pgpool47-extensions
• Config directory: /etc/sysconfig/edb/pgpool4.4/ -> /etc/sysconfig/edb/pgpool4.7/
• Binary directory: /usr/edb/pgpool4.4/bin/ -> /usr/edb/pgpool4.7/bin/
• Systemd service: edb-pgpool-4.4.service -> edb-pgpool-4.7.service

Test Case: Switchover

Objective: Validate a controlled switchover using the EFM promote -switchover command. The standby should be promoted to primary, the old primary should be reconfigured as a standby, and both EFM and PgPool should reflect the updated roles automatically.

Preconditions: EFM cluster is healthy, Primary and Standby databases are in sync, and PgPool is running with load balancing active.

Step 1: Check initial cluster status

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address              DB       VIP
    ----------------------------------------------------------------
    Primary     192.168.40.179       UP
    Standby     192.168.40.182       UP
    Witness     192.168.40.183       N/A

Allowed node host list:
    192.168.40.179 192.168.40.182 192.168.40.183
Membership coordinator: 192.168.40.179
Standby priority host list:
    192.168.40.182

DB Type     Address              WAL Received LSN   WAL Replayed LSN   Info
---------------------------------------------------------------------------
Primary     192.168.40.179                          0/5001290
Standby     192.168.40.182       0/5001290          0/5001290
Standby database(s) in sync with primary. It is safe to promote.

Step 2: Verify node roles in PgPool before switchover

enterprisedb$ /usr/edb/as16/bin/psql -p 9999 -h 192.168.40.1 -d edb -U enterprisedb \
  -c "show pool_nodes"

 node_id |    hostname    | port | status |  role   | select_cnt | load_balance_node | last_status_change
---------+----------------+------+--------+---------+------------+-------------------+--------------------
 0       | 192.168.40.179 | 5444 | up     | primary | 0          | false             | 2024-09-17 12:09:20
 1       | 192.168.40.182 | 5444 | up     | standby | 0          | true              | 2024-09-17 12:09:20

Step 3: Perform the switchover

Run efm promote efm -switchover from any EFM node (typically the current primary). EFM will promote the highest-priority standby and then reconfigure the old primary as a standby automatically.

root# /usr/edb/efm-4.9/bin/efm promote efm -switchover
Promote/switchover command accepted by local agent. Proceeding with promotion and will
reconfigure original primary. Run the 'cluster-status' command for information about
the new cluster state.

Step 4: Verify EFM cluster status post-switchover

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address              DB       VIP
    ----------------------------------------------------------------
    Standby     192.168.40.179       UP
    Primary     192.168.40.182       UP
    Witness     192.168.40.183       N/A

Allowed node host list:
    192.168.40.179 192.168.40.182 192.168.40.183
Membership coordinator: 192.168.40.179
Standby priority host list:
    192.168.40.179

DB Type     Address              WAL Received LSN   WAL Replayed LSN   Info
---------------------------------------------------------------------------
Primary     192.168.40.182                          0/60001B8
Standby     192.168.40.179       0/60001B8          0/60001B8
Standby database(s) in sync with primary. It is safe to promote.

Step 5: Verify updated node roles in PgPool post-switchover

EFM calls efm_pgpool_functions During the switchover, detach the old primary and reattach it as a standby. PgPool's view of the cluster should now reflect the swapped roles:

enterprisedb$ /usr/edb/as16/bin/psql -p 9999 -h 192.168.40.1 -d edb -U enterprisedb \
  -c "show pool_nodes"

 node_id |    hostname    | port | status |  role   | select_cnt | load_balance_node | last_status_change
---------+----------------+------+--------+---------+------------+-------------------+--------------------
 0       | 192.168.40.179 | 5444 | up     | standby | 1          | true              | 2024-09-17 12:14:29
 1       | 192.168.40.182 | 5444 | up     | primary | 0          | false             | 2024-09-17 12:14:29

Result: The switchover is successful. The new primary is 192.168.40.182 and the former primary 192.168.40.179 is now a standby. Both EFM and PgPool reflect the updated roles. The cluster is in sync and ready for another switchover or failover.

Test Case: Failover

Objective: Simulate a primary database failure. Verify that EFM automatically promotes the standby to primary, PgPool updates its backend node roles, and the cluster can be restored by rebuilding the failed node as a new standby.

Preconditions: EFM cluster is healthy with Primary UP, Standby UP, and Witness N/A. PgPool watchdog quorum is maintained.

Step 1: Verify initial state: PgPool watchdog and pool_nodes

root# /usr/edb/pgpool4.4/bin/pcp_watchdog_info -p 9898 -U enterprisedb -v
Password:
Watchdog Cluster Information
Total Nodes              : 3
Remote Nodes             : 2
Alive Remote Nodes       : 2
Quorum state             : QUORUM EXIST
Leader Node Name         : 192.168.76.16:9999 Linux pg03p8
Leader Host Name         : 192.168.76.16

Watchdog Node Information
Node Name         : 192.168.76.16:9999  Status: LEADER      Node priority: 3
Node Name         : 192.168.76.14:9999  Status: STANDBY     Node priority: 1
Node Name         : 192.168.76.15:9999  Status: STANDBY     Node priority: 2

enterprisedb$ psql -p 9999 -h 192.168.76.16 -d edb -U enterprisedb -c "show pool_nodes"
 node_id |   hostname    | port | status |  role   | select_cnt | load_balance_node
---------+---------------+------+--------+---------+------------+-------------------
 0       | 192.168.76.11 | 5444 | up     | primary | 0          | true
 1       | 192.168.76.12 | 5444 | up     | standby | 0          | false

Step 2: Verify EFM cluster status before failover

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address           DB    VIP
    ----------------------------------------------------------------
    Primary     192.168.76.11     UP
    Standby     192.168.76.12     UP
    Witness     192.168.76.13     N/A

Promote Status:
    Primary     192.168.76.11    WAL Replayed LSN: 0/110001B8
    Standby     192.168.76.12    WAL Received LSN: 0/110001B8  WAL Replayed LSN: 0/110001B8
    Standby database(s) in sync with primary. It is safe to promote.

Step 3: Simulate primary failure: stop the primary database

root# systemctl stop edb-as-16.service

Step 4: Observe EFM auto-promotion of the standby

Within seconds, EFM detects the primary failure and begins promoting the standby. The intermediate state shows the old primary as Idle and the standby as Promoting:

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address           DB       VIP
    ----------------------------------------------------------------
    Idle        192.168.76.11     UNKNOWN
    Promoting   192.168.76.12     UP
    Witness     192.168.76.13     N/A

Shortly after, the promotion completes and the cluster stabilises with the new primary:

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address           DB    VIP
    ----------------------------------------------------------------
    Idle        192.168.76.11     UNKNOWN
    Primary     192.168.76.12     UP
    Witness     192.168.76.13     N/A

Standby priority host list:
    (List is empty.)

Idle Node Status:
    192.168.76.11  UNKNOWN  Connection to 192.168.76.11:5444 refused.

Step 5: Verify PgPool reflects the new primary

EFM calls efm_pgpool_functions detach For the failed primary, the new primary is now identified by PgPool. Node 0 (the failed primary) is shown as down:

enterprisedb$ psql -p 9999 -h 192.168.76.16 -d edb -U enterprisedb -c "show pool_nodes"
 node_id |   hostname    | port | status |  role   | select_cnt | load_balance_node | last_status_change
---------+---------------+------+--------+---------+------------+-------------------+--------------------
 0       | 192.168.76.11 | 5444 | down   | standby | 0          | false             | 2025-05-09 10:40:08
 1       | 192.168.76.12 | 5444 | up     | primary | 0          | true              | 2025-05-09 10:40:33

Step 6: Rebuild the failed primary as a new standby

Clear the old data directory on the failed node and use pg_basebackup to clone from the new primary. The -R flag writes the standby.signal And connection info automatically.

enterprisedb$ rm -rf /var/lib/edb/as16/data/*

enterprisedb$ pg_basebackup -h 192.168.76.12 -D /var/lib/edb/as16/data/ \
  -R -P -Xs -U enterprisedb -p 5444
52174/52174 kB (100%), 1/1 tablespace

enterprisedb$ sudo systemctl start edb-as-16.service

Step 7: Verify the restored cluster

After starting the service, EFM detects the new standby and re-adds it to the cluster. PgPool re-attaches it as a load-balancing target:

root# /usr/edb/efm-4.9/bin/efm cluster-status efm
Cluster Status: efm
    Agent Type  Address           DB    VIP
    ----------------------------------------------------------------
    Standby     192.168.76.11     UP
    Primary     192.168.76.12     UP
    Witness     192.168.76.13     N/A

Promote Status:
    Primary     192.168.76.12    WAL Replayed LSN: 0/14000060
    Standby     192.168.76.11    WAL Received LSN: 0/14000060  WAL Replayed LSN: 0/14000060
    Standby database(s) in sync with primary. It is safe to promote.

enterprisedb$ psql -p 9999 -h 192.168.76.16 -d edb -U enterprisedb -c "show pool_nodes"
 node_id |   hostname    | port | status |  role   | select_cnt | load_balance_node | last_status_change
---------+---------------+------+--------+---------+------------+-------------------+--------------------
 0       | 192.168.76.11 | 5444 | up     | standby | 0          | false             | 2025-05-09 10:42:09
 1       | 192.168.76.12 | 5444 | up     | primary | 0          | true              | 2025-05-09 10:40:33

Result: The failover is complete. The new primary 192.168.76.12 is serving writes, the rebuilt node 192.168.76.11 Is streaming as a standby, and PgPool has updated its backend roles. The cluster is fully recovered and ready for production traffic.

Conclusion

This guide demonstrated a complete two-layer High Availability setup for EDB Postgres Advanced Server using EFM for database-level automatic failover and PgPool for connection pooling, load balancing, and PgPool-tier HA with watchdog. With this architecture, client applications connect to a single Delegate Virtual IP and receive automatic read/write routing; writes go to the primary, reads are load-balanced across primary and standby, with no manual intervention required on primary failure.

Key takeaways from the test cases:

• Switchover (planned): The efm promote efm -switchover Command gracefully swaps primary and standby roles. EFM automatically reconfigures the old primary as a standby and notifies PgPool via efm_pgpool_functions To update load balancing.
• Failover (unplanned): EFM detects a primary failure within seconds and promotes the standby. PgPool marks the failed backend as down and redirects all traffic to the new primary. The cluster is restored by running pg_basebackup on the failed node and restarting the EFM agent; no manual switchover command is needed.
• Upgrading to EFM 5.x: Replace auto.resume.period with auto.resume.startup.period and auto.resume.failure.period. Review detach.on.agent.failure As the default changed to false.
• Upgrading to PgPool 4.6+: Explicitly set sr_check_user, health_check_user, recovery_user, and wd_lifecheck_user  These now default to an empty string and will silently disable features if left unset.

Resources

• EDB Failover Manager Documentation (Latest)
• EDB PgPool Documentation (Latest)
• EDB Postgres Advanced Server Documentation
• PgPool-II Upstream Documentation