Overview of FlowServer for WarehousePG

FlowServer for WarehousePG (WHPG) provides a dedicated framework for your data ingestion pipelines. It is designed to deliver real-time visibility into the health and performance of ETL jobs, ensuring that data moving from streaming sources like Kafka and RabbitMQ reaches your WarehousePG cluster efficiently.

What it delivers

What it delivers

• High-performance parallel loading: Writes data delivered from clients directly into the segments of the WHPG cluster, bypassing coordinator bottlenecks for maximum throughput.

• Broad data source support: Seamlessly integrates with modern message brokers, supporting Kafka and RabbitMQ queues.

• Versatile format handling: Supports standard data formats including JSON, CSV, and Avro (Kafka only).

• Optimized command-line interface: Features a streamlined CLI that allows you to submit, start, list, and monitor jobs.

• Real-time job monitoring: Provides detailed visibility into active tasks, showing loading ratios, total row counts, and progress tracking.

• Native observability: Includes built-in Prometheus metrics to monitor system health and performance out of the box.

• Advanced stream control: Allows for precise data recovery and synchronization with flags to reset stream offsets to the earliest, latest, or specific timestamps.