Features of the Hadoop Foreign Data Wrapper v2

These are the key features of the Hadoop Foreign Data Wrapper.

WHERE clause pushdown

Hadoop Foreign Data Wrappper allows the pushdown of WHERE clauses to the foreign server for execution. This feature optimizes remote queries to reduce the number of rows transferred from foreign servers.

Column pushdown

Hadoop Foreign Data Wrapper supports column pushdown. As a result, the query brings back only those columns that are a part of the select target list.

Join pushdown

Hadoop Foreign Data Wrapper supports join pushdown. It pushes the joins between the foreign tables of the same remote Hive or Spark server to that remote Hive or Spark server, enhancing the performance.

For an example, see Example: Join pushdown.

Aggregate pushdown

Hadoop Foreign Data Wrapper supports aggregate pushdown. It pushes the aggregates to the remote Hive or Spark server instead of fetching all of the rows and aggregating them locally. This gives a very good performance boost for the cases where aggregates can be pushed down. The push-down is currently limited to aggregate functions min, max, sum, avg, and count, to avoid pushing down the functions that are not present on the Hadoop server. Also, aggregate filters and orders are not pushed down.

For more information, see Example: Aggregate pushdown.

Automated cleanup

Hadoop Foreign Data Wrappper allows the cleanup of foreign tables in a single operation using the DROP EXTENSION command. This feature is specifically useful when a foreign table is set for a temporary purpose. The syntax is:

DROP EXTENSION hdfs_fdw CASCADE;

For more information, see DROP EXTENSION.