Submitting the FlowServer job

Use FlowCLI to load data into WarehousePG (WHPG) using FlowServer. You can run FlowCLI from any host where you installed the FlowServer package and has connectivity to the FlowServer instance.

A typical command workflow when using FlowCLI to load data into WHPG is:

1. Submit a FlowServer job
2. Start the job
3. Monitor the job status
4. Stop the job
5. Remove the job

Refer to FlowCLI reference for more detailed command options.

Submitting a job

When you submit a job, you provide a YAML-formatted configuration file that defines the properties of the load operation. Load properties include WHPG-specific options, as well as properties that are specific to the data source.

The following example submits a FlowServer job named sample_upload whose load properties are defined in the configuration file named load_config.yaml:

flowcli submit --name sample_upload load_config.yaml

A newly-submitted FlowServer job is in the Submitted state.

Starting a job

Run the flowcli start subcommand to start a job. FlowServer initiates the data load operation from the client. It sets up the connection to WHPG and creates the external tables that it uses to load data directly into the segments.

The default behaviour of flowcli start is to return immediately. When you specify the --stop-at-eof option, the command reads data until it receives an EOF, and then stops the job. In this scenario, the job transitions to the Success or Error state when the command exits.

Example:

flowcli start sample_upload

A job that starts successfully enters the Running state.

Monitoring a job

Use the flowserver list subcommand to list all jobs and their status:

flowcli list

Use the flowserver monitor subcommand to check the progress of one or multiple jobs:

flocli monitor sample_upload

Stopping a job

Use the flowcli stop subcommand to stop a specific job. When you stop a job, FlowServer writes any unwritten batched data to the WHPG table and stops actively reading new data from the data source.

flowcli stop sample_upload

A job that you stop enters the Stopped state.

Removing a job

The flowcli remove subcommand removes a FlowServer job. When you remove a job, FlowServer unregisters the job from its job list and releases all job-related resources.

flowcli remove sample_upload