Loading data with FlowServer
FlowServer for WarehousePG (WHPG) supports loading data from Apache Kafka and RabbitMQ sources into a WHPG table.
Apache Kafka is a fault-tolerant, low-latency, distributed publish-subscribe message system. FlowServer for WHPG supports loading Kafka data from the Apache and Confluent Kafka distributions.
The FlowServer supports loading data from a RabbitMQ traditional AMQP queue into WHPG. FlowServer acts as a RabbitMQ consumer.
Perform the following tasks when you use the FlowServer to load data into a WHPG table:
- Ensure that you meet the prerequisites.
- Construct the load configuration file.
- Create the target table on WHPG and assign the necessary role permissions.
- Run the FlowCLI client to submit and manage the load job.
Prerequisites
Understand the prerequisites before loading data with FlowServer.
Constructing the configuration file
Learn how to construct the required yaml configuration file to load data to WarehousePG with FlowServer.
Creating the target table
Learn how to create the target table and configure role privileges to load data to WarehousePG with FlowServer.
Submitting the FlowServer job
Learn how to submit the Flowserver job to load data to WarehousePG.
Examples
Understand how to load data using Flowserver through the provided worked examples.
Could this page be better? Report a problem or suggest an addition!