Functions v1.10

The reference section is a list of functions available with Analytics Accelerator.

Refer to PGFS functions for Pipelines for details on how to create storage locations for PGAA.

Table functions

pgaa.list_analytics_tables()

Returns a list of all analytics tables managed by PGAA, including their configuration, storage statistics, and current replication status.

Synopsis

SELECT * FROM pgaa.list_analytics_tables();

Parameters

None.

Returns

A table can either have a storage location or a catalog. It cannot have both.

The catalog details refer to either the replication target, if performing a catalog-managed offload or replication to object storage, or to the source catalog location, if reading data from tables managed by an external catalog.

pgaa.lakehouse_table_stats()

Returns storage statistics for a specific analytical table, including the size of the latest active snapshot and the total cumulative size of all data versions (historical data and logs) stored in object storage.

Synopsis

SELECT * FROM pgaa.lakehouse_table_stats('table_name'::regclass);

Parameters

Returns

pgaa.test_storage_location()

Tests the connectivity and configuration of a defined storage location, and verifies read or write permissions.

Synopsis

SELECT pgaa.test_storage_location('location_name', test_writes:= true);

Parameters

Returns

Returns NULL if the test is successful. Returns a descriptive error message if the test fails.

pgaa.pgaa_version()

Returns the version information for the EDB Postgres Analytics Accelerator, including the specific build version, build date, and Git commit hash.

Synopsis

SELECT pgaa.pgaa_version();

Parameters

None.

Returns

A string containing the semantic version, the build timestamp, and the unique Git identifier for the current installation.

pgaa.engine_version()

Returns version information from the currently configured executor engine. Queries the backend engine selected by pgaa.executor_engine (Seafowl or Spark Connect) and returns its version details. For Seafowl, returns build and version metadata. For Spark Connect, returns the Spark version and, where available, Java, Scala, and OS information from the Spark driver.

Replaces pgaa.seafowl_version(), which was removed in PGAA 1.9.

Synopsis

SELECT pgaa.engine_version();

Parameters

None.

Returns

A string containing version details for the active executor engine.

Catalog functions

pgaa.add_catalog()

Registers a new Iceberg catalog with PGAA. This function performs an automated connection check to validate credentials and accessibility before the catalog is registered in the system.

Synopsis

SELECT pgaa.add_catalog('catalog_name', 'catalog_type', 'catalog_options'::json);

Parameters

Depending on which storage you use, your JSON file for the catalog_options must specify different options.

• For REST catalogs

  → WrapCopy

  {
  "url": "https://your-catalog-endpoint.com",
  "warehouse_name": "your_warehouse_name",
  "warehouse": "your_warehouse_id",
  "token": "your_secret_auth_token"
  "danger_accept_invalid_certs": "false"
  }

  Where:

  • url: The base HTTP(S) endpoint of the REST catalog service.

  • warehouse_name: A human-readable alternative to the warehouse ID, supported by some REST providers for easier configuration.

  • warehouse: The unique identifier for the specific warehouse within the catalog service.

  • danger_accept_invalid_certs: If set to true, Postgres skips SSL certificate validation. Use this only for internal testing or with self-signed certificates; never use it for sensitive public connections.

    Most REST catalogs vend short-term, table-scoped object store credentials to the client, so no further configuration is needed. If your REST catalog doesn't vend credentials, add a store_config object with the object store connection details:

    → WrapCopy

    {
    "url": "https://your-catalog-endpoint.com",
    "store_config": {
        "type": "s3",
        "bucket": "your-bucket-name",
        "access_key_id": "your-access-key",
        "secret_access_key": "your-secret-key",
        "region": "us-east-1",
        "endpoint": "https://your-object-store-endpoint.com",
        "allow_http": "true"
    }
    }

    Where:

  • store_config.type: The object store type, for example s3 for an S3-compatible store.

  • store_config.bucket: The name of the bucket holding the catalog's Iceberg tables.

  • store_config.access_key_id and store_config.secret_access_key: The credentials used to access the bucket.

  • store_config.region: (Optional) The object store region.

  • store_config.endpoint: (Optional) The object store endpoint, required for S3-compatible stores such as MinIO.

  • store_config.allow_http: (Optional) Set to "true" to allow non-TLS connections to the endpoint.

    store_config supports the same object store providers as PGFS. For the connection options for Azure or Google Cloud Storage, see the PGFS storage provider documentation.

    Some Iceberg REST Catalog providers, like Snowflake Open Catalog, might expect different parameters. Specify the following OAuth options:

    → WrapCopy

    {
    "oauth2.grant_type": "client_credentials",
    "oauth2.client_id": "oauth-client-id",
    "oauth2.client_secret": "oauth-client-secret",
    "oauth2.token_uri": "https://EXAMPLE-polaristest.snowflakecomputing.com/polaris/api/catalog/v1/oauth/tokens",
    "oauth2.scope": "PRINCIPAL_ROLE:read_only"
    }

    Where:

  • oauth2.grant_type: Specifies the OAuth2 flow. The most common value for service-to-service communication is client_credentials.

  • oauth2.client_id: The unique identifier for your application registered with the identity provider.

  • oauth2.client_secret: The secret key used to authenticate the client ID.

  • oauth2.token_uri: The full HTTP(S) endpoint where PGAA sends the credentials to request an access token.

  • oauth2.scope: (Optional) A space-delimited list of permissions or resources the client is requesting access to.

• For AWS S3 Tables:

  → WrapCopy

  {
  "arn": "arn:aws:s3tables:us-east-1:1234567890:bucket/my-bucket",
  "region": "us-east-1"
  }

  Where:

  • arn: Specifies the Amazon Resource Name (ARN), the unique identifier for your S3 Table bucket.
  • region: Specifies the physical AWS data center location where your S3 Table bucket resides.

Returns

Returns the name of the catalog on success.

pgaa.update_catalog()

Updates the configuration options (the JSON object) for an existing Iceberg catalog. Like pgaa.add_catalog(), this function performs a validation check to ensure the new connection parameters are functional before applying the changes.

Synopsis

SELECT * FROM pgaa.update_catalog('catalog_name', 'new_options'::json);

Parameters

See pgaa.add_catalog() for a detailed breakdown of the required JSON fields for each catalog type.

Returns

Returns the name of the catalog upon successful update.

Example

Rotate an OAuth2 token:

SELECT * FROM pgaa.lakehouse.update_catalog(
    'marketing_lakehouse', 
    '{
        "url": "https://iceberg.example.com",
        "oauth2.client_id": "pgaa_service_user",
        "oauth2.client_secret": "new_secure_secret_2026",
        "oauth2.token_uri": "https://auth.example.com/token"
    }'::json
);

pgaa.delete_catalog()

Removes a registered catalog from the database. To prevent accidental data loss, this function requires a cascade parameter to be explicitly set to true if there are still tables managed by the catalog.

Synopsis

SELECT * FROM pgaa.delete_catalog('catalog_name', cascade := true);

Parameters

Returns

Returns the name of the deleted catalog upon successful completion.

pgaa.list_catalogs()

Returns a list of all registered catalogs in the system, including their connection configuration, metadata synchronization timestamps, and current operational status.

Synopsis

SELECT * FROM pgaa.list_catalogs();

Parameters

None.

Returns

pgaa.import_catalog()

Performs a one-time scan and import of table definitions from a registered Iceberg catalog into Postgres. This function creates the local metadata required for PGAA to query the remote tables. This is a manual, once-off import and does not enable automatic, continuous synchronization.

Synopsis

SELECT pgaa.import_catalog('catalog_name', 'namespace_filter');

Parameters

Returns

None.

pgaa.attach_catalog()

Enables continuous metadata synchronization for a previously registered Iceberg catalog (using pgaa.add_catalog()). Once attached, PGAA automatically monitors the remote catalog for changes and updates the local Postgres metadata accordingly. See Catalog synchronization for polling rate configuration.

Synopsis

SELECT pgaa.attach_catalog('catalog_name');    

Parameters

Returns

None.

pgaa.detach_catalog()

Stops continuous metadata synchronization for a registered Iceberg catalog and moves it to a detached state. If cascade is true, it drops all tables managed by the catalog before detaching. If cascade is false, the tables remain in the database but synchronization is halted.

Synopsis

SELECT * FROM pgaa.detach_catalog('catalog_name', cascade := true);

Parameters

Returns

The function returns the row from the pgaa.catalog system table for the catalog being detached.

pgaa.test_catalog()

Tests the connectivity and configuration of a registered Iceberg catalog. This function verifies that the Postgres instance can communicate with the remote catalog endpoint and, optionally, validates that the provided credentials have write permissions.

Synopsis

SELECT pgaa.test_catalog('catalog_name', test_writes:=true);

Parameters

Returns

Returns NULL if the test is successful. Returns a descriptive error message if the test fails.

pgaa.list_catalog_tables()

Returns a list of all tables and views available in a registered Iceberg catalog. This function allows you to explore the contents of a remote catalog without having to import or attach the tables to your local database first.

Synopsis

SELECT * FROM pgaa.list_catalog_tables('catalog_name', 'namespace_filter');

Parameters

Returns

pgaa.drop_catalog_tables()

Removes all local Postgres table and view definitions that are managed by a specific Iceberg catalog, including any dependent objects if cascade is true. It does not affect tables in the remote Iceberg catalog.

Synopsis

SELECT * FROM pgaa.drop_catalog_tables('catalog_name', cascade := true);

Parameters

Returns

Spark functions

pgaa.spark_sql()

Executes a Spark SQL query directly on your Postgres cluster via the configured Spark Connect endpoint. This allows you to run Iceberg compaction routines or Spark functions that aren't available in Postgres.

See Spark procedures for a list of the available procedures.

To run this function, you must set the configuration parameter pgaa.spark_connect_url to point to an available Spark Connect service.

Synopsis

For a single catalog:

SELECT pgaa.spark_sql('query', 'catalog_name');

For multiple catalogs:

SELECT pgaa.spark_sql('query', ARRAY['catalog1', 'catalog2']);

Parameters

Returns

The result set of the Spark query formatted as a JSON object.

Example

Reduce metadata overhead via the rewrite_data_files Spark task:

SELECT pgaa.spark_sql($$  
            CALL preexisting.system.rewrite_data_files(  
              table => '"preexisting"."ns-1"."table-1"',  
              strategy => 'sort',  
              sort_order => 'value DESC',  
              options => map('rewrite-all', 'true')  
            )  
        $$);  

pgaa.execute_compaction()

Performs compaction to improve the performance and storage efficiency of an analytical table. Runs as a native background task and supports both Iceberg and Delta tables.

Synopsis

SELECT pgaa.execute_compaction('table_name'::regclass, 'settings'::json);

Parameters

Returns

None.

Example

SELECT pgaa.execute_compaction('lakehouse.inventory_iceberg'::regclass);

Background task functions

pgaa.launch_task()

Schedules a background maintenance task for an analytical Delta or Iceberg table.

Synopsis

SELECT pgaa.launch_task(
    'table_name'::regclass, 
    'task_type', 
    'task_options'::jsonb, 
    'scheduled_at'::timestamp
);

Parameters

The values for the JSONB task_options depend on each task_type. All options are optional except zorder, which requires columns, and purge, which requires both storage_location and path.

• compaction: Merges small files into larger ones to speed up analytical scans. The available task_options are:

  → WrapCopy

  {
  "target_size": 536870912,
  "preserve_insertion_order": true,
  "max_concurrent_tasks": 10,
  "max_spill_size": 2147483648,
  "min_commit_interval": 60,
  "dry_run": false
  }

  Where:

  • target_size (Delta only): Specifies the size of the output files in bytes.
  • preserve_insertion_order (Delta only): Whether to maintain the existing sort order of rows.
  • max_concurrent_tasks (Delta only): Limits the number of parallel tasks the executor can run.
  • max_spill_size (Delta only): Sets the maximum data size in bytes allowed to spill to disk during the process.
  • min_commit_interval (Delta only): Sets the minimum wait time in seconds between committing updates to the Delta log.
  • dry_run: If true, calculates and logs the operation, but does not perform compaction.

• zorder (Delta only): A clustering technique that reorganizes data across multiple columns to improve "data skipping" for queries with filters on those columns. The available task_options are:

  → WrapCopy

  {
  "columns": ["customer_id", "transaction_date"],        
  "target_size": 1073741824,
  "preserve_insertion_order": false,
  "max_concurrent_tasks": 4,
  "max_spill_size": 2147483648,
  "min_commit_interval": 30
  }

  Where:

  • columns: (Required) An array of strings representing the columns to be used for Z-ordering.
  • target_size: Specifies the size of the output files in bytes.
  • preserve_insertion_order: Whether to maintain the existing sort order of rows.
  • max_concurrent_tasks: Limits the number of parallel tasks the executor can run.
  • max_spill_size: Sets the maximum data size in bytes allowed to spill to disk during the process.
  • min_commit_interval: Sets the minimum wait time in seconds between committing updates to the Delta log.

• vacuum (Delta only): Deletes old data files that are no longer referenced by the Delta transaction logs, freeing up space in object storage. The available task_options are:

  → WrapCopy

  {
  "retention_period": "168 hours",
  "dry_run": false,
  "enforce_retention_duration": true
  }

  Where:

  • retention_period: Defines the age at which unreferenced files become eligible for deletion.
  • dry_run: If true, calculates and logs the files that would be deleted, but performs no deletions.
  • enforce_retention_duration: If true, the task validates the retention_period against the system's global minimum safety limit.

• purge (Delta only): Explicitly removes data from a specific storage path. The available task_options are:

  → WrapCopy

  {
  "storage_location": "s3_main",
  "path": "archive/2023/temp/"
  }

  Where:

  • storage_location: Required. The name of the storage location.
  • path: The relative directory path or file prefix within the storage location that should be permanently deleted.

Returns

A unique task ID for the task.

You can check the task status by querying the pgaa.background_task table and the provided task ID.

Examples

• Delete old data files prior to the last 7 days:

  → WrapCopy

  SELECT pgaa.launch_task(
      'sales.transactions', 
      'vacuum', 
      '{"retention_period": "7 days", "dry_run": false}'::jsonb
  );

• Perform compaction:

  → WrapCopy

  SELECT pgaa.launch_task(
      'telemetry.logs', 
      'compaction', 
      '{
          "target_size": 536870912, 
          "max_concurrent_tasks": 2
      }'::jsonb
  );

• Reorganize data rows by clustering on columns region and customer_id:

  → WrapCopy

  SELECT pgaa.launch_task(
      'crm.customers', 
      'zorder', 
      '{
          "columns": ["region", "customer_id"], 
          "target_size": 1073741824
      }'::jsonb
  );

Postgres Distributed (PGD) integration functions

pgaa.convert_to_analytics()

Converts a PGD-replicated table (HTAP) to the PGAA access method. This function points the table directly to the analytics data residing in object storage (Iceberg or Delta format) that has been synchronized via replication.

Synopsis

CALL pgaa.convert_to_analytics('relation_name'::regclass);

Parameters

Returns

None.

pgaa.convert_to_tiered_table()

Converts a heap table into a PGD-managed tiered table with automatic partitioning and analytics offloading. This function automates the creation of a partitioned staging table, migrates existing data, and configures the lifecycle policies for both hot and cold tiers. See Implementing tiered tables for details.

Synopsis

CALL pgaa.convert_to_tiered_table(
    relation := 'relation_name'::regclass,
    range_partition_column := 'column_name',
    partition_increment := 'interval_string',
    analytics_offload_period := 'interval',
    initial_lower_bound := 'start_date_string',
    retention_period := 'interval' DEFAULT NULL,
    enable_replication := boolean DEFAULT false,
    minimum_advance_partitions := integer DEFAULT 2,
    maximum_advance_partitions := integer DEFAULT 5,
    drop_after_retention_period := boolean DEFAULT true,
    purge_analytics_target := boolean DEFAULT false
);

Parameters

Returns

None.

Example

Converts a heap table into a tiered structure where data is partitioned by month, replicated to the cloud for real-time analytics, and automatically offloaded to cold storage once it is older than three months.

CALL pgaa.convert_to_tiered_table(
    relation                    := 'public.sensor_readings'::regclass,
    range_partition_column      := 'reading_time',
    partition_increment         := '1 month',
    analytics_offload_period    := '3 months',
    initial_lower_bound         := '2026-01-01',
    retention_period            := '2 years',
    enable_replication          := true,
    minimum_advance_partitions  := 3,
    maximum_advance_partitions  := 6,
    drop_after_retention_period := true,
    purge_analytics_target      := true
);

pgaa.disable_analytics_replication()

Disables PGD replication to object storage for an HTAP table. This function reverts the table from an HTAP state back to a Heap state by stopping the continuous replication stream to object storage. See Replicating to analytics for details.

Synopsis

CALL pgaa.disable_analytics_replication('relation_name'::regclass);

Parameters

Returns

None.

pgaa.enable_analytics_replication()

Enables PGD replication for a heap table to object storage for analytics. This function transitions a standard Heap table to an HTAP table by initiating a continuous replication stream to the analytics engine. See Replicating to analytics for details.

Synopsis

CALL pgaa.enable_analytics_replication(
    relation regclass,
    purge_analytics_target boolean DEFAULT false
);

Parameters

Returns

None.

pgaa.htap_table_stats_uncached()

Returns statistics for HTAP tables in PGD environments.

Synopsis

SELECT * FROM pgaa.htap_table_stats_uncached(
    htap_catalog text,
    htap_catalog_namespace text,
    htap_catalog_table text,
    htap_storage_location text,
    htap_storage_path text,
    htap_format text
);

Parameters

Returns

pgaa.list_tiered_tables()

Returns detailed information about PGD-managed tiered tables, including their partition settings, data retention policies, analytics offloading configurations, and storage size metrics.

Synopsis

SELECT * FROM pgaa.list_tiered_tables();

Parameters

None.

Returns

pgaa.restore_from_analytics()

Converts a PGAA analytics table residing in object storage back to the standard PostgreSQL heap access method.

Synopsis

CALL pgaa.restore_from_analytics('relation_name'::regclass);

Parameters

Returns

None.

Utility functions

pgaa._bench_replication()

Runs a synthetic, pgbench-like replication benchmark to measure the performance of data ingestion into a specified storage location or catalog. It calculates the initial data load speed and the sustained transaction throughput.

Synopsis

SELECT * FROM pgaa._bench_replication(
    storage_location text DEFAULT NULL,
    catalog text DEFAULT NULL,
    format text DEFAULT 'iceberg',
    scale integer DEFAULT 100,
    transactions integer DEFAULT 100000,
    dry_run boolean DEFAULT false
);

Parameters

Returns