Viewing metrics and logs from AWS
BigAnimal sends all metrics and logs from Postgres clusters to AWS. You can view metrics and logs directly from AWS using AWS CloudWatch.
Logs
AWS CloudWatch
When BigAnimal deploys workloads on AWS, the logs from the Postgres clusters are forwarded to AWS CloudWatch log groups as log streams. To query BigAnimal logs, you can use filter and patterns syntax or the CloudWatch Logs Insights section.
Query Postgres cluster logs
All logs from your Postgres clusters are presented as log streams in CloudWatch log groups. To find the CloudWatch log groups:
Sign in to the AWS console.
From Services, select CloudWatch.
Select the region corresponding where you choose to deploy your BigAnimal cluster.
Under Logs, select Log groups in the menu on the left.
The following log groups and log streams are available in the Customer CloudWatch space.
| Log group name | Log stream name | Description | Logger |
|---|---|---|---|
| /biganimal/PostgresLogs | from-pg_cluster_logs_for_customer | Logs of the Customer clusters databases (all Postgres-related logs) | logger = postgres |
| /biganimal/PostgresAuditLogs | from-pg_cluster_audit_logs_for_customer | Audit Logs of the Customer clusters databases | logger = pgaudit or edb_audit |
You can use the CloudWatch Logs Insights query syntax to compose your queries over these log streams.
Logs are split into structured fields matching those of the Postgres
csvlog format
with a record. prefix. For example, the sql_state_code is in the record.sql_state_code log field. The pg_cluster_id field identifies the specific Postgres cluster that originated the log message.
For example, with the /biganimal/PostgresLogs log group selected in CloudWatch Logs Insights, you can view all log entries with the following CloudWatch Logs Insights query:
fields record.log_time, record.error_severity, record.message | sort record.log_time desc
With the /biganimal/PostgresAuditLogsog log group selected, you can view the pgaudit and edb_audit logs with:
fields record.log_time, record.error_severity, record.audit.statement | filter logger == "pgaudit" | sort record.log_time desc fields record.log_time, record.error_severity, record.message | filter logger == "edb_audit" | sort record.log_time desc
Metrics
The BigAnimal instances collect a variety of metrics and make them available to your AWS subscription for dashboarding, alerting, querying, and other analytics. Most of these metrics are acquired directly from Postgres system tables, views, and functions. The Postgres documentation serves as the main reference for these metrics.
Available metrics
The following log group in AWS CloudWatch contains metrics specific to BigAnimal.
| Log group name | Log stream name | Description |
|---|---|---|
| /biganimal/Telemetry | from-http.0 | Metrics streams from BigAnimal Prometheus. |
You can use the CloudWatch Logs Insights query language to compose queries that analyze time-series metrics over these log groups and report the latest samples of metrics. Query the from-http.0 log stream in the /biganimal/Telemetry log group.
The available set of metrics is subject to change. Metrics might be added, removed, or renamed. Where feasible, we will change the metric name when the meaning or type of an existing metric changes.
Currently, all metrics forwarded from Prometheus are in the /biganimal/Telemetry log group. This might change in a future release.
Metrics labels
All Postgres metrics share a common labeling scheme. Entries generally have at least the following labels.
| Name | Description |
|---|---|
| instance | IP address of the host the metric originated from |
| postgresql | BigAnimal postgres cluster identifier, e.g., p-abcdef012345 |
| role | Postgres instance role: primary or standby replica |
| datname | Postgres database name (where applicable) |
| pod | k8s pod name |
| aks_cluster_name | AKS cluster name |
The labels field of a metrics entry is a nested field under the JSON-typed Message field.
To query the field for individual values, you dot reference (Message.labels) the labels object.
Example usage:
With /biganimal/Telemetry log group selected:
fields Message.labels.postgresql,Message.name, Message.value | filter strcontains (Message.name, 'cnp') and Message.labels.role == 'primary' | sort @timestamp desc
See Metrics for a detailed list of available metrics.
Dashboards
You can view logs and metrics from your Postgres clusters using AWS CloudWatch dashboards.
To access AWS CloudWatch dashboards:
Sign in to the AWS console.
From Services, select CloudWatch.
Select the region where you choose to deploy your BigAnimal cluster.
In the CloudWatch menu on the left, select Dashboards.
Select Telemetry Dashboard to launch a CloudWatch dashboard for your deployed clusters.
- On this page
- Logs
- Metrics
- Dashboards