Customizing Migration Portal secrets for secure internal communication Innovation Release

Suggest edits

Required for deployments with the migration installation scenario enabled. These secrets manage internal communication between the Migration Portal, the AI Copilot, and the underlying databases.

While edbctl can generate these automatically, production environments often require manual overrides for specific service accounts:

Database accounts: Internal service users and superuser credentials.

Copilot authentication: HTTP and Metrics authentication for secure component inter-communication.

While our standard installation guide already includes instructions on using edbctl to generate these custom secrets, this page provides a deeper dive, which includes the manual creation of secrets as an alternative and how to override existing secrets to update or rotate your credentials.

Creating Migration Portal secrets using edbctl

Note

Required for deployments with the migration installation scenario enabled. This scenario is included by default unless it is manually excluded via the spec.scenarios parameter in values.yaml.

Create custom secrets required for the Migration Portal:

  • For manual installations, run this command and follow the interactive prompts:

    edbctl setup create-install-secrets --version <version> --scenario migration

  • If you are running the installation via a CI/CD pipeline, you must suppress interactive prompts. The method for achieving this depends on your edbctl version:

    1. Configure edbctl for non-interactive behavior:

      edbctl config set interactive_mode off
edbctl config set confirm_mode off

    2. Run the setup command:

      edbctl setup create-install-secrets --version <version> --scenario migration

This creates the namespaces, custom secrets, and generates secure random passwords for all five service accounts:

  • edb-migration-portal.db_secrets
  • edb-migration-portal.db_superuser_secrets
  • edb-migration-portal.copilot_secrets
  • edb-migration-copilot.db_secrets
  • edb-migration-copilot.metrics_auth_secrets

For more information, see Understanding Migration Portal secrets.

Overriding existing secrets manually

If you need specific usernames or passwords, you can create the secrets manually. For production environments, we recommend using appropriate secure processes that don’t expose the data to your terminal.

Creating secrets

  1. Create a function that generates secure passwords for each of the service user accounts:

    function generate_pw {
 dd if=/dev/urandom bs=32 count=1 2>/dev/null | base64 | tr ‘+/’ ‘-_’ | tr     -d ‘=’
}

  2. Set environment variables for all the users and passwords required by your secrets:

    HTTP_USERNAME=test_user_001
HTTP_PASSWORD=$(generate_pw)
DB_SUPERUSER_USERNAME=postgres
DB_SUPERUSER_PASSWORD=$(generate_pw)
PORTAL_DB_USERNAME=test_user_002
PORTAL_DB_PASSWORD=$(generate_pw)
COPILOT_DB_USERNAME=test_user_003
COPILOT_DB_PASSWORD=$(generate_pw)
METRICS_USERNAME=test_user_004
METRICS_PASSWORD=$(generate_pw)

  3. Create the namespaces where you will store the custom secrets:

    kubectl create namespace edb-migration-copilot
kubectl create namespace edb-migration-portal

  4. Create the secret that determines the database user credentials:

    kubectl create secret generic custom-db-secrets \
  --namespace=edb-migration-portal \
  --from-literal=username="${PORTAL_DB_USERNAME}" \
  --from-literal=password="${PORTAL_DB_PASSWORD}" \
  --type=kubernetes.io/basic-auth

  5. Create the secret that determines the database superuser credentials:

    kubectl create secret generic custom-db-superuser-secrets \
  --namespace=edb-migration-portal \
  --from-literal=username="${DB_SUPERUSER_USERNAME}" \
  --from-literal=password="${DB_SUPERUSER_PASSWORD}" \
  --type=kubernetes.io/basic-auth

  6. Create the secret that authenticates HTTP communication between the copilot and Migration Portal:

    kubectl create secret generic custom-edb-migration-copilot-auth \
  --namespace=edb-migration-portal \
  --from-literal=username=${HTTP_USERNAME} \
  --from-literal=password=${HTTP_PASSWORD} \
  --type=kubernetes.io/basic-auth

  7. Create the secret that provides the copilot with access to the Migration Portal database:

    kubectl create secret generic custom-ragchew-db-secrets \
  --namespace=edb-migration-copilot \
  --from-literal=username="${COPILOT_DB_USERNAME}" \
  --from-literal=password="${COPILOT_DB_PASSWORD}" \
  --type=kubernetes.io/basic-auth

  8. Create the secret that enables monitoring of the copilot instance:

    kubectl create secret generic custom-edb-migration-copilot-metrics-auth \
  --namespace=edb-migration-copilot \
  --from-literal=username=${METRICS_USERNAME} \
  --from-literal=password=${METRICS_PASSWORD} \
  --type=kubernetes.io/basic-auth

Apply the secrets to your installation

To override the default secrets that come with the Hybrid Manager installation, reference the new secrets in either the values.yaml or HybridControlPlane custom resource (depending on the installation method you used):

parameters:
  edb-migration-copilot:
    db_secrets: custom-ragchew-db-secrets
    metrics_auth_secrets: custom-edb-migration-copilot-metrics-auth
  edb-migration-portal:
    copilot_secrets: custom-edb-migration-copilot-auth
    db_owner: ${PORTAL_DB_USERNAME}
    db_secrets: custom-db-secrets
    db_superuser_secrets: custom-db-superuser-secrets

Continue with the installation process. Once you perform the bootstrapping process by either invoking the helm upgrade command to apply the values.yaml config file or apply the HybridControlPlane, the installation will take care of using the secret overrides.

If you're overriding the secrets after installation, ensure you bootstrap again to make the modifications effective.

Back up the secrets

You can fetch the secrets like this: 

kubectl get secret -n <namespace> <secret_name> -o yaml

This command prints the secret's contents in YAML format, which you can then copy and store safely and securely. Repeat this with each of the created secrets.

Understanding Migration Portal secrets

For each secret, Description explains why you need it. Helm value shows where in the configuration file to reference it. Parameters shows on how to set non-default values.

See Overriding the default secrets for an example of how to configure custom secrets and override the default ones.

Migration Portal database account

Description: Configures a database service account used by Hybrid Manager to store Migration Portal system data and schema assessment results into a dedicated database.

Helm value: parameters.edb-migration-portal.db_secrets

apiVersion: v1
kind: Secret
metadata:
  name: <db-secret-name>
  namespace: edb-migration-portal
stringData:
  username: "<db username>" 
  password: "<db password>" 
type: kubernetes.io/basic-auth

Parameters:

  • metadata.name: Name of the secret you will later reference in the Helm value.
  • stringData.username: User that connects to the database. Also enter this user in the parameters.edb-migration-portal.db_owner Helm value.
  • stringData.password: Assign a generated password for this user.

Migration Portal database superuser account

Description: Configures a database superuser account used by Hybrid Manager to provision Migration Portal and the copilot databases.

Helm value: parameters.edb-migration-portal.db_superuser_secrets

apiVersion: v1
kind: Secret
metadata:
  name: <db-superuser-secret-name>
  namespace: edb-migration-portal
stringData:
  username: "postgres"
  password: "<db superuser password>"
type: kubernetes.io/basic-auth

Parameters:

  • metadata.name: Name of the secret you will later reference in the Helm value.
  • stringData.username: Must always be postgres.
  • stringData.password: Assign a generated password for this user.

Migration Copilot HTTP access from Migration Portal

Description: Defines the credentials used to authenticate HTTP communication between the copilot and Migration Portal.

Helm value: parameters.edb-migration-portal.copilot_secrets

apiVersion: v1
kind: Secret
metadata:
  name: <copilot-auth-secret>
  namespace: edb-migration-portal
stringData:
  username: "<http username>" 
  password: "<http password>"
type: kubernetes.io/basic-auth

Parameters:

  • metadata.name: Name of the secret used to authenticate HTTP requests that you will later reference in the Helm value.
  • stringData.username: Choose a username for Migration Portal to use when authenticating with the copilot.
  • stringData.password: Assign a generated password for this user.

Migration Copilot and Migration Portal database account

Description: Allows the copilot to communicate with the Migration Portal database. On initialization, the edb-migration-copilot component ensures this user is created and that correct ownership and permissions are set in the copilot database.

Helm value: parameters.edb-migration-copilot.db_secrets

apiVersion: v1
kind: Secret
metadata:
  name: <copilot_mp_secret_name>
  namespace: edb-migration-copilot
stringData:
  username: "<db username>"
  password: "<db password>"
type: kubernetes.io/basic-auth

Parameters:

  • metadata.name: Name of the secret used to provide the copilot with access to the Migration Portal database.
  • stringData.username: Choose a username for the copilot to use when accessing the Migration Portal database.
  • stringData.password: Assign a generated password for this user.

Monitoring and Migration Copilot

Description: Allows Hybrid Manager to monitor the state of the copilot deployment via Grafana.

Helm value: parameters.edb-migration-copilot.metrics_auth_secrets

apiVersion: v1
kind: Secret
metadata:
  name: <copilot-metrics-secret-name>
  namespace: edb-migration-copilot
stringData:
  username: <metrics-http-username> 
  password: <metrics http password>
type: kubernetes.io/basic-auth

Parameters:

  • metadata.name: Name of the secret used to allow monitoring of the copilot.
  • stringData.username: Set this to a username different from the one configured for Migration Copilot HTTP access from Migration Portal.
  • stringData.password: Assign a generated password for this user.
← Prev

Customizing Lakehouse/Catalog secrets

↑ Up

Customizing secrets

Next →

Implementing TLS certificates