So just to give a brief summary of this tool first; the EDB*Loader tool is a high-performance bulk data loader that provides an Oracle compatible interface for EDB Postgres Advanced Server. The EDB*Loader command line utility loads data from an input source, typically a file, into one or more tables using a subset of the parameters offered by Oracle SQL Loader.
We went through an exercise testing out the EDB*Loader tool with different Postgres database AS versions {9.4, 9.6,10.1} to show the speed of this high-performance bulk data loader going from 5 million up to 100 million rows which were all under 6 mins load time. We automated the bulk load & captured timings; first doing a truncate of the table before each load & a bounce of the Database cluster after each load to avoid data being cached, to get accurate timings. Some other things to point out; we were on a docker image container ( version 18.06.1-ce-win73 ) on Centos 6 on our laptop PC (16GB RAM and CPU 2.7GHz speed), using default database memory configuration, no indexes on the table, & we used a conventional load method with no parallelism which still shows how fast these timings are with such a large amount of data loaded. We did see some timing differences around 35,45, & 50 million rows loaded but overall witnessed quite impressive load speeds. Also, on the 10.1 version we had archive log mode on in which we still were seeing fast load times. We thought this was a great story to tell showing the speed of our high-performance data loader tool and we have graphed out each load amounts vs. timings as well. We have also attached the EDB*Loader excel sheet where we captured the timings on a chart for viewing as well... In conclusion, as you can see, this linear graph between each version is pretty close to scale.
Rows Loaded | 9.4 | 9.6 | 10.1 |
5000000 | 00:00:13.5 | 00:00:11.9 | 00:00:09.2 |
10000000 | 00:00:26.7 | 00:00:17.7 | 00:00:18.3 |
15000000 | 00:00:39.2 | 00:00:28.0 | 00:00:28.0 |
20000000 | 00:00:51.3 | 00:00:35.6 | 00:00:40.4 |
25000000 | 00:01:07.7 | 00:00:45.6 | 00:00:46.2 |
30000000 | 00:01:15.8 | 00:00:54.8 | 00:01:14.9 |
35000000 | 00:01:45.6 | 00:01:08.1 | 00:01:37.2 |
40000000 | 00:02:00.0 | 00:01:23.3 | 00:02:01.5 |
45000000 | 00:02:26.4 | 00:01:53.5 | 00:02:06.5 |
50000000 | 00:02:43.6 | 00:01:57.7 | 00:02:34.6 |
55000000 | 00:03:22.1 | 00:02:17.4 | 00:02:41.7 |
60000000 | 00:03:44.4 | 00:02:30.0 | 00:03:13.3 |
70000000 | 00:03:16.0 | 00:03:11.1 | 00:02:46.2 |
80000000 | 00:03:52.8 | 00:03:30.4 | 00:03:48.8 |
90000000 | 00:04:21.1 | 00:04:02.2 | 00:04:30.9 |
100000000 | 00:05:02.0 | 00:04:57.7 | 00:05:28.3 |