Known issues v1.4.2 (LTS)

These are the currently known issues and limitations identified in the Hybrid Manager (HM) Innovation Releases. Workarounds can sometimes help you mitigate the impact of these issues. These issues are actively tracked and will be resolved in a future release.

Multi-DC

Cluster creation fails on AWS multi-datacenter deployments with TDE enabled

Tip

Resolved in HM versions 1.4.2 and later.

Description: In AWS multi-datacenter environments, creating the primary and replica clusters together with Transparent Data Encryption (TDE) enabled returns an error, and cluster creation fails.

Workaround: Create the primary cluster with TDE enabled first. After the primary cluster is up and running, add the replica cluster with TDE enabled.

Core platform and resources

Cross-stream upgrade issues for OpenShift users

Tip

Resolved in HM versions 1.4.0 and later.

Description: Due to the OpenShift Operator Lifecycle Manager (OLM) enforcing strict semantic versioning, date-based versions (like 2026.1) are interpreted as being significantly "higher" than standard versions (like 1.4). This can prevent the OLM from recognizing a move to an LTS version as an upgrade.

Workaround: There is no workaround available for this issue. We are currently decoupling the Operator versions from the HM application versions to resolve this.

IdP configuration fails after upgrade to version 2026.2

Description: In some cases, upgrading from version 2026.1 to 2026.2 can cause your Identity Provider (IdP) configuration to display the Expiring Soon status in the HM console. This is caused by a race condition during the migration process. While this issue doesn't occur in every environment, it prevents users from successfully authenticating via their IdP provider if triggered.

Workaround: To resolve this state and allow the configuration to restore correctly, a cluster administrator must manually update the internal application database.

Execute the following SQL statement on the internal app-db database before attempting to re-setup the IdP integration in the HM console:

UPDATE upm_api_admin.idp SET deleted_at = NOW() WHERE deleted_at IS NULL;

No kapp-controller support in Hybrid Manager app to pull images

Description: The kapp-controller doesn't support Managed Identity/IAM Roles for pulling images.

Workaround: Users must manually configure the PackageRepository using a SecretRef (Kubernetes Secret) for registry authentication, as cloud-native managed identities are currently not supported.

SPIRE agent fails to start on RKE2 with SELinux enabled

Description: On RKE2 clusters running on RHEL (or other Linux distributions with SELinux in enforcing mode), the SPIRE agent pod enters a CrashLoopBackOff state with the error create UDS listener: listen unix /tmp/spire-agent/public/spire-agent.sock: bind: permission denied. This occurs because SELinux blocks the container process from creating a Unix domain socket on the HostPath volume /run/spire/agent-sockets. Environments without SELinux (such as Ubuntu-based RKE2 clusters) aren't affected.

Workaround: Disable SELinux on all RKE2 cluster nodes. For this, set SELINUX=disabled in /etc/selinux/config and reboot the nodes. After reboot, verify that getenforce shows Disabled. The SPIRE agent pods will recover automatically, though pods already in CrashLoopBackOff state may take several minutes due to the backoff timer.

Database cluster engine

Failure to create 3-node PGD cluster when max_connections is non-default

Description: Creating a 3-data-node PGD cluster fails if the configuration parameter max_connections is set to a non-default value during initial cluster provisioning.

Workaround: Create the PGD 3-data-node cluster using the default max_connections value. Update the value after the cluster is successfully provisioned.

HA clusters use verify-ca instead of verify-full for streaming replication certificate authentication

Description: Replica clusters use the less strict verify-ca setting for streaming replication authentication instead of the recommended, most secure verify-full. This is currently necessary because the underlying CloudNativePG (CNP) clusters don't support IP subject alternative names (IP SANs), which are required for verify-full in certain environments (like GKE load balancers).

Workaround: None. A fix depends on the underlying CNP component supporting IP SANs.

EDB Postgres Distributed (PGD) cluster with 2 data groups and 1 witness group not healthy

Tip

Resolved in HM versions 1.4.0 and later.

Description: PGD clusters provisioned with the topology of two data groups and one witness group may fail to reach a healthy state upon creation. This failure is caused by an underlying conflict between the bdr extension (used for replication) and the edb_wait_states extension. The combination of these two extensions in this particular topology prevents the cluster from initializing successfully.

CNPG operator enters an infinite reconcile loop when bootstrap Job creation is interrupted

Description: If a cluster reconciliation is interrupted after the PVC is created but before the bootstrap Job is created — for example, due to an optimistic locking conflict — subsequent reconciles enter an infinite loop. The operator waits for the PVC to reach a Ready status, but this can never occur without the Job, causing the loop to continue indefinitely.

Workaround: Delete the affected PVC to allow the operator to recreate it and break the loop. Because this issue only affects newly created PVCs, the PV won't contain any data yet and it's safe to delete.

kubectl delete pvc <pvc-name> -n <cluster-namespace>

Backup and recovery

Volume snapshot restoration is limited to the same region

Description: While HM automatically handles cross-region data synchronization for replica clusters using the internal backup mechanism, manual restoration of a cluster from a volume snapshot doesn't support cross-region operations. If you attempt to restore a cluster to a location in a different region using a volume snapshot, the operation will fail because snapshots are geographically restricted to their source region.

Workaround: To restore data to a location in a different region, use a Barman backup instead of a volume snapshot. Barman backups are accessible across regions.

WAL archiving is slow due to default parallel configuration

Description: The default setting for wal.maxParallel is too restrictive, which slows down WAL archiving during heavy data loads. This can cause a backlog of ready-to-archive WAL files, potentially leading to disk-full conditions. This parameter isn't yet configurable on the HM console.

Workaround: Manually edit the objectstores.barmancloud.cnpg.io Kubernetes resource for the specific backup object store and increase the wal.maxParallel value (for example, to 20) to accelerate archiving.

transporter-db disaster recovery (DR) process may fail due to WAL gaps

Tip

Resolved in HM versions 1.4.0 and later.

Description: The DR process for the internal transporter-db service may fail when restoring from the latest available backup. This occurs in low-activity scenarios where a backup was completed, but no subsequent write-ahead log (WAL) file was archived immediately following that backup. This gap prevents the restore process from successfully completing a reliable point-in-time recovery.

Workaround: To ensure a successful restore, select an older backup to restore that has at least one archived WAL file immediately following it. This makes the needed transactional logs available for the recovery process.

Full cluster recovery fails with pg_ctl: server does not shut down error

Description: Under race conditions, restoring into a cluster with HA architecture that is created and managed by HM may fail. The error logs on the full-recovery pods will contain entries similar to the following:

pg_ctl: server does not shut down
pg_ctl: waiting for server to shut down............................................................... failed
controller: Error while deactivating instance err=error stopping PostgreSQL instance: exit status 1

Workaround: Restore into a single-node cluster first. After the restore completes, expand the cluster to HA architecture.

Agent Factory and model management

Failure to deploy nim-nvidia-nvclip model with profile cache

Description: Model creation for the nim-nvidia-nvclip model fails in the Agent Factory when the profile cache is used during the deployment process.

Workaround: An administrator must manually download the necessary model profile from the NVIDIA registry to a local machine. They must then upload the profile files directly to HM's object storage path. Then they deploy the model by patching the Kubernetes InferenceService YAML with a specific environment variable to force it to use the pre-cached files instead of attempting a failed network download.

Workaround details
  1. Log in to the NVIDIA Container Registry (nvcr.io) using your NGC API key:

    docker login nvcr.io -u '$oauthtoken' -p $NGC_API_KEY
  2. Pull the Docker image to your local machine:

    docker pull nvcr.io/nim/nvidia/nvclip:latest
  3. Prepare a local directory for the downloaded profiles:

    mkdir -p ./model-cache
    chmod -R a+w ./model-cache
  4. Select the profile for your target GPU.

    For example, A100 GPU profile: 9367a7048d21c405768203724f863e116d9aeb71d4847fca004930b9b9584bb6

  5. Run the container to download the profile. The container is run in CPU-only mode (NIM\_CPU\_ONLY=1) to prevent GPU-specific initialization issues on the download machine.

    export NIM_MANIFEST_PROFILE=9367a7048d21c405768203724f863e116d9aeb71d4847fca004930b9b9584bb6
    export NIM_CPU_ONLY=1
    
    docker run -v ./model-cache:/opt/nim/.cache -u $(id -u) -e NGC_API_KEY -e NIM_CPU_ONLY -e NIM_MANIFEST_PROFILE --rm nvcr.io/nim/nvidia/nvclip:latest

    This container doesn't exit. You must manually stop the run (Ctrl+C) after you see the line Health method called in the logs, which confirms the profile download is complete.

  6. Upload the profiles from your local machine to the object storage bucket used by your HM deployment:

    gcloud storage cp -r ./model-cache gs://uat-gke-edb-object-storage/model-cache/nim-nvidia-nvclip
    Note

    Adjust the gs:// path to match your deployment's configured object storage location.

  7. Create the model nim-nvidia-nvclip using the HM console, specifying the Model Profiles Path field as the previous location (for example, /model-cache/nim-nvidia-nvclip). The deployment will initially fail or become stuck.

  8. Export the InferenceService YAML from the HM Kubernetes cluster.

  9. Add the necessary environment variable, NIM_IGNORE_MODEL_DOWNLOAD_FAIL, to the env section of the spec.predictor.model block in the exported YAML. This flag tells the NIM container to use the locally available cache (the files you uploaded) and ignore the network download failure.

    # --- Snippet of the modified InferenceService YAML ---
    spec:
      predictor:
        minReplicas: 1
        model:
          modelFormat:
            name: nim-nvidia-nvclip
          name: ""
          env:
          - name: NIM_IGNORE_MODEL_DOWNLOAD_FAIL  # <-- ADD THIS LINE
            value: "1"                         # <-- ADD THIS LINE
          resources:
            # ... resource requests/limits ...
          runtime: nim-nvidia-nvclip
          storageUri: gs://uat-gke-edb-object-storage/model-cache/nim-nvidia-nvclip
    # ---------------------------------------------------
  10. Apply the modified YAML using kubectl to force the deployment to use the pre-downloaded profiles:

    kubectl apply -f <modified-inference-service-file.yaml> -n <model-cluster-namespace>

The pods now start successfully, using the model profiles you manually provided using object storage.


Model configuration settings reset after Innovation Release upgrade

Tip

HM now uses Langflow instead of Griptape, so this issue is no longer applicable.

Description: After upgrading HM from the 2025.11 to the 2025.12 Innovation Release, the existing model configuration settings are unintentionally reset to empty. This results in a model_not_found error, preventing access to AI services and causing issues like knowledge bases (KBs) failing to display in the HM console.

Workaround: In the HM console, manually reenter or set your required model configurations to restore functionality.

Error page appears when editing knowledge base credentials

Tip

HM now uses Langflow instead of Griptape, so this issue is no longer applicable.

Description: When editing a knowledge base (KB) that was created from a pipeline in the HM console, skipping the username field and immediately navigating to the password field triggers a client-side JavaScript error (Cannot read properties of null), which results in an Unexpected Application Error! page.

Workaround: To prevent the error page from appearing, fill in the Username field immediately after selecting Edit KB and before attempting to enter the password.

Missing aidb_users role on PGD witness node prevents AI functionality

Description: The aidb_users role (necessary for Agent Factory functionality) and its related extension aren't being successfully replicated to the witness node of a PGD cluster during cluster initialization. This issue is specific to PGD clusters that use a single witness node (as opposed to a witness group) and results in the Agent Factory encountering errors due to the missing required role.

Workaround: To manually install the necessary role and allow the Agent Factory to function, execute the following SQL commands directly on the PGD witness node:

begin;
set local bdr.ddl_replication = off;
SET LOCAL bdr.commit_scope = 'local';
create user aidb_users;
commit;

Pipeline Designer-created knowledge bases not visible to GenAI Builder

Description: In HM 2025.12, knowledge bases (KBs) created with Pipeline Designer (PD) are assigned to the visual_pipeline_user role. This is done to enforce strict isolation, ensuring HM users can't access SQL-created KBs (and vice versa) by default. However, this isolation prevents GenAI Builder from querying these KBs out of the box.

Workaround: Explicitly share PD-created KBs with your specific PostgreSQL user account to make them queryable in GenAI Builder.

Workaround details

Example scenario

Alice connects to HM as alice@acme.org and to PostgreSQL as the database user alice. She must share the PD KBs with the alice database user to enable GenAI Builder agents to query them using her credentials.

  1. Prerequisite: Ensure user existence

    Ensure the target PostgreSQL user (alice in this example) exists and is assigned the aidb_users role. If the user doesn't exist, execute the following (assuming EDB documentation was followed for AIDB installation):

    CREATE USER alice WITH PASSWORD '********';
    GRANT CONNECT ON DATABASE <some_db> TO alice;
    GRANT CREATE ON SCHEMA <some_schema> TO alice;
    GRANT aidb_users TO alice;
  2. Grant role access (the workaround)

    To allow the user to view and query PD-created knowledge bases, grant them membership in the visual_pipeline_user role:

    GRANT visual_pipeline_user TO alice;
  3. Configure

    Update the GenAI Builder agent configuration to use the alice credentials (username and password) you set.


Pipeline creation fails on PGD clusters with AIDB v6.1

Tip

Resolved in HM versions 1.4.0 and later, which ships with AIDB 7.3.2.

Description: In HM version 2026.3, when using Agent Factory with EDB Postgres Distributed (PGD) and AIDB v6.1, creating a pipeline on a replicated table may fail. You may encounter the following error: ERROR: replicated relation <table_name> cannot have triggers that are owned by roles other than the relation owner (SQLSTATE 42501).

This occurs because PGD requires that both the trigger and its associated trigger handler functions be owned by the same role that owns the table (typically visual_pipeline_user). In version 2026.3, these functions may be assigned to a different owner by default.

Workaround: To resolve this ownership conflict, a superuser must manually transfer ownership of the AIDB trigger handler functions to the visual_pipeline_user role.

Execute the following SQL commands on the affected database:

ALTER FUNCTION aidb.pipeline_background_trigger_handler OWNER TO visual_pipeline_user;
ALTER FUNCTION aidb.pipeline_live_trigger_handler OWNER TO visual_pipeline_user;

External cloud-hosted OCR models fail to register via the HM console

Description: The aidb.sync_hcp_models function automatically appends a /v1/infer suffix to all model URLs during synchronization. While this suffix is required for local containerized NIM models, it breaks connectivity for external cloud-hosted models such as NVIDIA's PaddleOCR (which uses a URL path like .../cv/baidu/paddleocr). As a result, external OCR models cannot be successfully configured or used via the HM console.

Workaround: Bypass the HM console registration flow and register the model directly in the database using SQL via the aidb.create_model function, providing the correct URL explicitly.

Option 1 — use the built-in default URL for remote NVIDIA models:

SELECT aidb.create_model(
    'my_ocr_model',
    'nim_paddle_ocr',
    credentials => jsonb_build_object('api_key', current_setting('aidb.nvidia_nim_api_key', true)),
    replace_credentials => true
);

Option 2 — explicitly set a custom full URL:

SELECT aidb.create_model(
    'my_ocr_model',
    'nim_paddle_ocr',
    config => '{"url":"https://ai.api.nvidia.com/v1/cv/baidu/paddleocr"}'::JSONB,
    credentials => jsonb_build_object('api_key', current_setting('aidb.nvidia_nim_api_key', true)),
    replace_credentials => true
);

Analytics and tiered tables

PGAA causes a server crash (SIGSEGV) when executing JOIN queries with constant result relations in PostgreSQL 17

Tip

Resolved in HM versions 1.4.0 and later.

Description: When the Postgres Analytics Accelerator (PGAA) extension is loaded, executing a JOIN query where one side is a constant result relation — for example, SELECT aaa FROM v_dual LEFT JOIN bbb or SELECT aaa FROM (SELECT 1) d LEFT JOIN bbb — causes a SIGSEGV crash. This issue is due to a type-confusion bug in PostgreSQL 17, where three different path subtypes share the same T_Result pathtype.

Workaround: Disable join and aggregate pushdown in PGAA:

ALTER DATABASE <your_database> SET pgaa.enable_join_pushdown    = off;
ALTER DATABASE <your_database> SET pgaa.enable_groupby_pushdown = off;

If you're using a Distributed High Availability (PGD) cluster, replicate the settings to all nodes:

SELECT bdr.run_on_all_nodes($$
ALTER DATABASE <your_database> SET pgaa.enable_join_pushdown    = off;
ALTER DATABASE <your_database> SET pgaa.enable_groupby_pushdown = off;
$$);

HM console and observability

GUC settings aren't applied until after a restart

Tip

Resolved in HM versions 1.4.0 and later.

Description: GUC settings configured in the HM console, such as shared_preload_libraries, aren't applied immediately. The settings remain in their previous state until at least one restart has been performed.

Workaround: After applying GUC changes in the HM console, trigger a restart to ensure the new settings take effect.

Asset Library apps require Project Owner or Project Editor role

Description: All Asset Library apps require users to have at least the Project Editor role to access, deploy, or manage them. Users with lower-privilege roles, such as Project Viewer, can't access Asset Library apps. This role requirement applies to all Asset Library apps and isn't currently configurable on a per-app basis.

Workaround: Ensure that any user who needs to access or manage Asset Library apps is assigned at least the Project Editor role for the relevant project. Role assignments can be managed by a Project Owner or organization owner in the project settings.

Metabase fails to deploy when using an EPAS cluster as its database backend

Description: Deploying Metabase from the Asset Library fails when the selected database cluster is an EDB Postgres Advanced Server (EPAS) instance. Metabase enters a CrashLoopBackOff state with a Liquibase migration error: ERROR: syntax error at or near "$". This occurs because Liquibase identifies EDB Postgres Advanced Server as a different database type than community PostgreSQL. As a result, property definitions scoped to dbms="postgresql" (such as ${TEXT.TYPE} and ${TIMESTAMP_TYPE}) are skipped, while dependent changesets still execute with unresolved placeholders, causing the SQL syntax error.

Workaround: Use a community PostgreSQL cluster as Metabase's database backend instead of EDB Postgres Advanced Server. EDB Postgres Advanced Server clusters can still be connected to Metabase as data sources for analytics after deployment.

Asset Library apps pgAdmin4 and Apache Superset fail to deploy on RHOS

Tip

This issue was resolved for Apache Superset in HM versions 1.4.0 and later. And for pgAdmin4 in HM versions 1.4.1 and later.

Description: In Red Hat OpenShift (RHOS) environments, deploying the pgAdmin4 and Apache Superset OSS Library applications failed due to OpenShift Security Context Constraint (SCC) violations. The upstream images for these apps ran as specific user IDs (for example, 5050 for pgAdmin4 and 0 for Apache Superset's init container) that fell outside the allowed UID ranges enforced by OpenShift's restricted SCC policies.

Workaround: In versions before HM 1.4.1, avoid deploying pgAdmin4 or Apache Superset in RHOS environments. Upgrade to HM 1.4.1 or later to resolve the issue.

Superset deployment fails with DuplicateTable error

Description: Deploying Superset from the Asset Library may fail with a DuplicateTable error: psycopg2.errors.DuplicateTable: relation "idx_user_id" already exists. This occurs when the selected backend database already contains tables from a previous Superset deployment.

Workaround: Use a new, empty database when deploying Superset.

Asset Library app CPU and memory metrics include failed and completed pod states

Tip

Resolved in HM versions 1.4.0 and later.

Description: The CPU and memory usage metrics displayed for Asset Library apps include pods in Failed or Completed states, such as pods spawned by jobs. These metrics are only available during the app's startup phase, so the displayed figures may not accurately reflect the app's current resource usage until the metrics stabilize.

Workaround: Wait 30 minutes after app startup for the metrics to stabilize and reflect the correct state.

Asset Library apps may get stuck pending on RHOS when storage classes differ

Tip

Resolved in HM versions 1.4.2 and later.

Description: On Red Hat OpenShift (RHOS) environments where the cluster default storage class differs from the spec.globalParameters.storage_class value — for example, where the default is lvms-vm (backed by control-plane nodes) but the configured storage class is lvms-bm (backed by worker nodes) — Asset Library apps that require persistent volumes, such as pgAdmin4, may get stuck in a Pending state indefinitely. This occurs because the app's PVC inherits the cluster default storage class, which may not have capacity available on the nodes where app pods can be scheduled.

Workaround: On environments where an app deployment gets stuck, delete the pending PVC in the app namespace and recreate it with storageClassName set to the value from hcp.spec.globalParameters.storage_class. Alternatively, change the cluster default StorageClass to match hcp.spec.globalParameters.storage_class so all new PVCs automatically use the correct storage.

OSS Library app deployment fails on GKE ARM64 when the requested storage size is less than 4Gi

Description: On GKE node pools running the arm64 n4a (Google Axion) machine family, legacy Persistent Disks are unsupported, requiring a switch to a hyperdisk-balanced StorageClass. This transition introduces a strict Google Cloud infrastructure constraint: a hard minimum volume size of 4Gi is enforced on all hyperdisk-balanced provisions.

If you deploy an OSS Library app (for example pgBadger, pgAdmin 4 etc.) and request a persistent volume smaller than 4Gi, provisioning fails at the storage layer. The following happens:

  • The app's PersistentVolumeClaim (PVC) stays in a Pending state indefinitely.

  • The app never reaches the Ready status in Estate > Apps.

  • Describing the PVC (kubectl describe pvc <pvc-name> -n <app-namespace>) shows a ProvisioningFailed event with an error similar to:

    failed to provision volume with StorageClass "hyperdisk-balanced":
    rpc error: code = InvalidArgument desc = ... requested size ... is less than the minimum required size 4Gi

This issue only affects GKE node pools running the arm64 n4a (Google Axion) machine family with hyperdisk-balanced storage class. AMD64 (x86_64) node pools and other cloud providers aren't affected.

Workaround: Before deploying an OSS Library app on a GKE ARM64 cluster, set the app's persistent volume size parameter to 4Gi or larger:

If a deployment has already failed because of this issue, delete the app and redeploy it with a storage size of 4Gi or greater.

HammerDB app doesn't become healthy on GKE ARM64 due to missing ARM64 toleration

Description: On Google Kubernetes Engine (GKE) clusters that use arm64 node pools, the HammerDB OSS Library app can't be scheduled onto ARM64 worker nodes. GKE ARM64 nodes are tainted (typically with kubernetes.io/arch=arm64:NoSchedule) so that only pods that explicitly tolerate ARM64 architecture are scheduled onto them. The HammerDB pod spec shipped with the OSS Library package currently doesn't include this toleration.

As a result, when you deploy HammerDB on a GKE ARM64 cluster:

  • The HammerDB pod stays in a Pending state.

  • The HammerDB app never reaches the Ready status in Estate > Apps.

  • Running kubectl describe pod <hammerdb-pod> -n <app-namespace> shows a scheduling event similar to:

    0/N nodes are available: N node(s) had untolerated taint {kubernetes.io/arch: arm64}. 
    preemption: 0/N nodes are available: N Preemption is not helpful for scheduling.

This behavior is specific to GKE ARM64 clusters. Other OSS Library apps and other architectures (AMD64/x86_64) aren't affected.

Workaround: There's no supported workaround at this time. Don't deploy the HammerDB OSS Library app on GKE ARM64 clusters until this issue is resolved. If you need to run HammerDB benchmarks against a PostgreSQL cluster hosted on GKE ARM64, deploy the HammerDB app onto a project that runs on an AMD64 node pool and point it at the target PostgreSQL cluster using its connection string.

HammerDB app image doesn't support ARM64 architecture

Description: The HammerDB OSS Library app image is built for the amd64 (x86_64) architecture only. On clusters that use arm64 node pools, HammerDB can't run because no arm64 build of the image exists yet. This is a separate, deeper issue than the ARM64 scheduling problem described above: even once a HammerDB pod can be scheduled onto an arm64 node, the image itself isn't able to execute there. The limitation stems from an upstream HammerDB dependency that doesn't yet support building for arm64.

Workaround: There's no workaround at this time. Deploy the HammerDB OSS Library app onto a project that runs on an AMD64 (x86_64) node pool. If the target Postgres cluster runs on arm64 infrastructure, point HammerDB at it using its connection string instead of deploying HammerDB on the same arm64 cluster.

Clusters and upgrades

Upgrade to version 2026.4 fails due to immutable fluent-bit-k8s-events StatefulSet

Description: When upgrading to HM 2026.4, the upgrade process may fail with a Failed to install component error for upm-fluent-bit. This occurs because the upgrade attempts to modify an immutable field in the fluent-bit-k8s-events StatefulSet spec, which the Kubernetes API server rejects. The error message is: StatefulSet.apps "fluent-bit-k8s-events" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'ordinals', 'template', 'updateStrategy', 'persistentVolumeClaimRetentionPolicy' and 'minReadySeconds' are forbidden.

Workaround: Before upgrading to 2026.4, manually delete the fluent-bit-k8s-events StatefulSet in the edb-observability namespace. The operator will recreate it with the correct configuration during the upgrade:

kubectl delete statefulset fluent-bit-k8s-events -n edb-observability

Agent Factory

Knowledge base data isn't replicated to non-primary nodes on PGD clusters

Tip

Resolved in HM versions 1.4.0 and later.

Description: When using Agent Factory with AIDB 7.3.0 on a PGD cluster, knowledge base data (visible via the aidb.knowledge_bases_v6 view) is only present on the primary node and isn't automatically replicated to other nodes. This occurs because the aidb.knowledge_base_registry and aidb.knowledge_base_pipeline tables, introduced in AIDB 7, weren't included in the PGD setup script.

Workaround: Before creating any knowledge bases, manually run the following SQL commands on the PGD cluster:

SELECT bdr.alter_sequence_set_kind('aidb.knowledge_base_registry_id_seq'::regclass, 'galloc', 1);
SELECT bdr.replication_set_add_table('aidb.knowledge_base_registry');
SELECT bdr.replication_set_add_table('aidb.knowledge_base_pipeline');

Index analysis actions fail due to missing hypopg extension

Description: The Agent Factory actions analyze_workload_indexes and analyze_query_indexes are currently unavailable because the required hypopg extension is missing from the standard HM database images. Attempting to run these actions will result in an error indicating the extension is not installed.

Workaround: There is currently no manual workaround to install this extension on managed clusters. Avoid using these two specific actions until the extension is included in a future image update.

Missing validation for required MCP tool parameters

Description: Several actions within the MCP tools allow fields to be left empty during configuration, which results in execution errors rather than validation warnings. If these parameters are not populated, the workflow may fail to build or return an "Unsupported object type" error.

Affected actions and parameters:

  • list_objects and get_object_details: The Object Type field must not be empty.

  • explain_query: The Analyze and hypothetical_indexes fields must be populated with specific data types.

Workaround: Ensure the following parameters are manually filled before executing the flow:

  • list_objects (Object Type): Enter table, view, sequence, function, stored procedure, or extension.

  • get_object_details (Object Type): Enter table, view, sequence, or extension.

  • explain_query (Analyze): Enter True or False.

  • explain_query (hypothetical_indexes): Use tool mode to provide a valid list.

Some EDB Component tools are non-functional due to incompatible API request formatting

Description: Some tools within the EDB Component are currently non-functional due to incompatible API request formatting. By default, the component exposes too many tools, which can exceed API limits and trigger errors.

Workaround: Enable tool mode and apply specific filters to limit the number of tools exposed by the component.

Text search may match on non-visible fields

Description: When using the search functionality in the HM console, for example in the Activity Log section, the system performs a full-text search across all underlying data fields. As a result, search results may include resources or activity log entries where the visible Activity Name or other shown attributes don't appear to match the search term.

Workaround: There is currently no workaround for this behavior as it is a result of the underlying search design. Users should be aware that search results are inclusive of internal action details not displayed in the primary log view.

Agent Studio MCP server components not supported in HM Flow Hosting

Description: Flows that include an Agent Studio MCP (Model Context Protocol) server component fail when executed via the Hybrid Manager Flow Hosting API with the following error: Error building Component my_mcp: Error updating tool list: Langflow MCP server functionality is not available. This feature requires the full Langflow installation. The HM Flow Hosting runtime (lfx-runtime) is a lightweight environment that does not include the full Agent Studio installation. Flows using MCP server components will appear to deploy successfully — the pod reports as healthy — but will fail at runtime when invoked via the REST API.

Workaround: Avoid including Agent Studio MCP server components in flows intended for deployment via the HM Flow Hosting API. If you need to run the Airman or HM MCP servers, use the EDB Airman or EDB Platform components instead — these components are compatible with the lfx-runtime environment. For any other MCP server functionality, use the full Agent Studio interactive environment instead.

Running a test query against an empty knowledge base returns a 500 error

Tip

Resolved in HM versions 1.4.0 and later.

Description: When running a test query from the Knowledge Bases list view against a knowledge base whose underlying embeddings table is empty — for example, while the KB pipeline is still in a Processing state — the API returns an HTTP 500 error: ERROR: Query returned no data. Hint: The "<pipeline_name>" table is likely empty. The API should return an empty result set instead.

Workaround: Wait until the knowledge base pipeline finishes processing before running test queries. The pipeline status changes from Processing to an active state once data has been indexed and the embeddings table is populated.

EDB Model Server component causes agent failures when used with the EDB Airman MCP component

Tip

Resolved in HM versions 1.4.0 and later.

Description: When building an agent flow in Agent Studio that uses the EDB Model Server component together with the EDB Airman MCP component, the agent may fail with an error similar to: messages with role 'tool' must be a response to a preceding message with 'tool_calls'. This issue is intermittent and doesn't occur when using other model provider components.

Workaround: Use the OpenAI component or the generic model component in place of the EDB Model Server component. Flows using these alternative model providers aren't affected by this issue.

Updated Agent Studio flow changes aren't immediately reflected in HM after republishing

Description: When you edit a previously published Agent Studio flow in the HM Agent Studio editor and republish it to the Asset Library, the updated version doesn't immediately appear in HM. There is a delay of up to 5 minutes before the changes propagate and become visible.

Workaround: After republishing a flow, wait approximately 5 minutes before verifying that the changes are reflected in HM.

Migrations

Known issues pertaining to the HM Migration Portal, data migration workflows, and schema ingestion workflows are maintained on a dedicated page in the Migrating databases documentation. See Known issues, limitations, and notes for a complete list.