Installing ClickHouse and PostgreSQL (optional)

Install ClickHouse to store the historical metrics and log data the OTel Collector forwards from every cluster node, and an optional PostgreSQL database for WEM's own configuration and application state. You must install ClickHouse before WEM, since WEM's configuration depends on knowing its address.

ClickHouse isn't bundled with WEM. The steps on this page cover one convenient path to a working instance using EDB's ClickHouse packages, but you can also point WEM at an existing ClickHouse instance you already manage, on any host reachable from every cluster node's OTel Collector or gateway collector. See ClickHouse's own documentation for sizing, replication, and other production configuration this page doesn't cover.

You can install ClickHouse:

On the standby coordinator: the simplest topology for a single WHPG cluster. If your cluster has no standby coordinator, install it on the coordinator instead.
On a dedicated monitoring host, outside the cluster: keeps observability components isolated from the cluster's own nodes.

Regardless of topology, install the gateway collector only when multiple systems or clusters send observability data to the same ClickHouse instance. A single WHPG cluster sending data directly to ClickHouse doesn't need one.

Installing ClickHouse

Install ClickHouse on your host of choice:

Configure the EDB repository, then install the ClickHouse packages:

export EDB_SUBSCRIPTION_TOKEN=<your-token>
export EDB_REPO=clickhouse
curl -1sSLf "https://downloads.enterprisedb.com/$EDB_SUBSCRIPTION_TOKEN/$EDB_REPO/setup.rpm.sh" | sudo -E bash
sudo yum install -y edb-clickhouse-server edb-clickhouse-client

Modify the contents /etc/clickhouse-server/config.xml by adding your settings as a drop-in file under config.d/, which ClickHouse merges in automatically. Fragments are only merged at startup, so this setting has no effect until you restart ClickHouse.
Set the following parameters:
- Listen address (listen_host): the network interface ClickHouse binds to. Required, since ClickHouse listens on localhost only by default and needs to accept connections from the gateway collector and from WEM.
- HTTP port (http_port): the port the gateway collector uses for OTLP ingest over HTTP. Already 8123 by default, ClickHouse's own default and the value WEM expects, so no change is needed unless you want a different port.
- TCP port (tcp_port): the port for ClickHouse's native protocol, used by clickhouse-client and by WEM. Already 9000 by default, so likewise no change is needed unless you want a different port.
- Logger: log level and destination. Optional, the defaults already write to the same log paths, just at a more verbose level than this configuration sets.
Set all four in a single drop-in file:
```
sudo mkdir -p /etc/clickhouse-server/config.d
sudo tee /etc/clickhouse-server/config.d/observability.xml << 'EOF'
<clickhouse>
    
    <listen_host>0.0.0.0</listen_host>

    
    

    
    

    <logger>
        <level>information</level>
    </logger>
</clickhouse>
EOF
```
Set a password for the default ClickHouse user. This example uses ClickHouse's built-in default user, but any ClickHouse user works, as long as you use the same username consistently in the gateway collector and WEM configuration:
```
sudo tee /etc/clickhouse-server/users.d/default-password.xml << 'EOF'
<clickhouse>
    <users>
        <default>
            <password><your-password></password>
        </default>
    </users>
</clickhouse>
EOF
```

Start ClickHouse:

sudo systemctl daemon-reload
sudo clickhouse start
sudo clickhouse status

Verify ClickHouse is accepting connections:

clickhouse-client --password "<your-password>" --query "SELECT 1"

Confirm that ClickHouse is listening on all interfaces:

grep -A1 listen_host /var/lib/clickhouse/preprocessed_configs/config.xml
ss -tlnp | grep -E ':8123|:9000'

Test the connection from another cluster node rather than from ClickHouse's own host:
```
curl "http://<clickhouse-host>:8123/?query=SELECT%201"
```

Installing PostgreSQL (optional)

WEM needs a Postgres database to store its own configuration and application state, dashboards, alert rules, and user accounts, separate from the WHPG cluster it monitors. You can host this data on a dedicated instance, or reuse your WHPG cluster's own Postgres server instead. See Application state for the trade-offs and the role to create either way.

To set up a dedicated instance:

Download and install PostgreSQL from the official downloads page, or install it using your system package manager.
Initialize the database, then enable and start the PostgreSQL service. See the PostgreSQL documentation for details.
On PostgreSQL 13 and earlier, password_encryption defaults to md5, which is incompatible with the scram-sha-256 authentication method this procedure's pg_hba.conf step uses. Check the setting, and fix it if needed, before creating the role:
```
sudo -i -u postgres psql -c "SHOW password_encryption;"
sudo -i -u postgres psql -c "ALTER SYSTEM SET password_encryption = 'scram-sha-256';"
sudo -i -u postgres psql -c "SELECT pg_reload_conf();"
```
Create the role WEM connects with. See Application state for the exact privileges required:
```
CREATE ROLE wem_admin WITH LOGIN PASSWORD '<your-password>' CREATEDB;
```
Configure pg_hba.conf to allow wem_admin to authenticate with a password, using all for the database field since wem setup connects to the postgres maintenance database before switching to wem:
```
host    all    wem_admin    127.0.0.1/32    scram-sha-256
host    all    wem_admin    ::1/128         scram-sha-256
```
Verify PostgreSQL is accepting connections:
```
pg_isready -h <postgres-host> -p 5432
```

Next steps

If your deployment needs the gateway collector, continue to Installing the gateway collector.

Otherwise, continue to Installing WEM.