FlowCLI reference

Manage the WarehousePG FlowServer jobs. It allows users to submit, start, stop, and monitor jobs.

Synopsis

flowcli <subcommand> [global options]

Subcommands

Submit

Submit a job by providing a YAML configuration file or a directory. If a directory is provided, all YAML files in that directory will be submitted as separate jobs.

Command usage

flowcli submit <jobconfig.yaml | /directory/path> [options]

The subcommand submit supports the following options:

• -n | --name <job_name>: (Optional) Specify a name for the job. If not provided, the job name will be derived from the configuration file name.
• -s | --start: (Optional) Immediately start the job after submission. If specified, you also can add the options for the start subcommand.

Example:

flocli submit -n kafka_job kafka.yaml

Start

Start a job or multiple jobs.

Command usage

flowcli start <job_name> [options]

The subcommand start supports the following options:

• --all: Start all jobs that are not in running state.
• --stop-at-eof: Stop the job when the end of the input data is reached. This is applicable for jobs that read from a source like Kafka or RabbitMQ.
• --reset-to-earliest: Reset to the earliest offset in the source on job starting. This is applicable for jobs that read from Kafka.
• --reset-to-latest: Reset to the latest offset in the source on job starting. This is applicable for jobs that read from Kafka.
• --reset-to-timestamp <timestamp>: Reset to the specified timestamp in the source on job starting. This is applicable for jobs that read from Kafka.
• --skip-explain: Skip the SQL explain phase of the job on starting. This is helpful for reducing the startup time of the job.

Stop

Stop a job or multiple running jobs.

Command usage

flowcli stop [<job_name>] [options]

The subcommand stop supports the following options:

• --all: Stop all running jobs

List

List all jobs with their status and details.

Command usage:

flowcli list [options]

The subcommand list supports the following options:

• -n | --name-only: Only show the names of the job, and hide other details.

Monitor

Monitor the progress of a job or multiple jobs.

Command usage:

flowcli monitor <job_name>

Global options

The following options can be used with any of the flowcli subcommands:

• --host <host>: The hostname or IP address of the WarehousePG Flowserver. Default is localhost.
• --port <port>: The port number on which the WarehousePG Flowserver is running. Default is 6060.
• --log-dir <log_directory>: The directory where the logs will be stored. Default is $HOME/gpAdminLogs.