Contributed by Jason Davis
EnterpriseDB and Lenovo have been working together to promote a powerful yet cost-effective choice for customers of Lenovo’s X6 line of enterprise servers. EDB’s engineering team got hold of a Lenovo 3850 X6 and decided to put it through the paces with Postgres Plus Advanced Server (PPAS) 9.4 and a pre release version of the 9.5 database. The benchmark results were remarkable.
To see how the system would perform and compare to other tests EDB has run, we decided to make use of pgbench to execute a few different test cycles. Pgbench is a simple program for running benchmark tests on EDB Postgres Plus or PostgreSQL. It runs a workload that is loosely based on TPC-B, involving five SELECT, UPDATE and INSERT commands per transaction. Pgbench allows you to specify ‘scale factors’, which tell the system how many rows of data to create. At the default "scale factor" of 1, the tables initially contain this many rows:
table # of rows
------------------------------------------
pgbench_branches 1
pgbench_tellers 10
pgbench_accounts 100000
pgbench_history 0
Increasing the scale factor makes the database larger. This allows us to test different sizes of databases to see how the system performs when all the data can fit into Postgres’s shared_buffers vs. how it performs when it can’t fit in shared_buffers but still fits into system memory. We typically test with scale factors of 100 (~1.5 gb), 300 (~4.5 gb), 1000 (~15 gb) and 3000 (~45 gb) to see the differences.
More information about pgbench is available on the postgresql.org web site.
Hardware & configuration details of the Lenovo 3850 X6:
- 4x Intel® Xeon® E7-4890 v2 (15 cores, 37.5M Cache, 2.80 GHz) processors
- 512GB RAM (32x 16GB 1600 MHz DDR3 DIMMs)
- 4 x 300GB SAS 10K RPM Hard Drives configured with ServeRAID M5210 w/ RAID 1
- Storage is partitioned and has logical volumes as follows:
- lv_home —> /home 223GB, lv_root —> / 50GB, lv_swap — /swap 4GB, tmpfs —> /dev/shm 505GB, devpts —/dev/pts, sysfs —/sys, proc -/proc
- lv_home attr - wi-ao 223GB, lv_root attr - wi-ao 50GB, lv_swap attr - wi-ao 4GB
Given the configuration of this system – a cutting edge CPU and memory, but a non-optimal disk configuration for processing writes to the database – we saw this as an opportunity to benchmark read intensive application workloads using the pgbench -S option.
Following are the two graphs we generated to show the performance of the system under various scale factors for PPAS 9.4 and PPAS 9.5-development:
While there are numerous insights we can see from these graphs, there are two particular observations that I want to highlight here:
Raw Performance:
- The raw performance is incredible. Peak throughputs of over 600k tps on PPAS 9.4 and over 700k tps on PPAS 9.5-devel are the largest we have seen on any hardware by a good margin.
- A few months earlier, a colleague of ours ran similar pgbench tests on an alternative hardware platform having 24 cores and 192 hardware threads with 512GB RAM. While it’s not an apples-to-apples comparison (the 3850 X6 has 60 cores and 120 hardware threads), it is a similar class of hardware.
- A scale factor of 300 on the alternative system resulted in a peak performance of 300k on 9.4 & 500k on 9.5-devel (compared to 600k & 700k on Lenovo).
- A scale factor of 1000 on the alternative system resulted in peak performance of 250k on 9.4 and 350k on 9.5-devel (compared to 330k & 675k on Lenovo).
Improvements of PPAS 9.5-devel over 9.4:
- The tests with PPAS 9.5-devel show great improvement over 9.4 for a database with a scale factor 1,000 (which is almost twice the size of the shared buffers), where it reaches better performance than the 9.4 run with a 300 scale factor that entirely fits into shared buffers.
- The tests with PPAS 9.5-devel also show large improvements at a scale factor 3,000, where we observe peak performance to be almost twice what it was in 9.4
- It was surprising to see that the scale factor 3000 on 9.5-devel did have a decrease in performance after 80-90 concurrent clients, and that is something we are planning to investigate further.
- The tests also show continued improvement in tps beyond 140 clients and up to 220 or 230 clients, depending on the scale factor.
- This is mainly due to some work that EDB and others in the open source community did for PostgreSQL 9.5 to relieve low level lock contention at a few critical points within the system. Specifically, commit - ab5194e6 (Improve LWLock scalability) written by Andres Freund and reviewed by EnterpriseDB’s Amit Kapila and Robert Haas, commit id 5d7962c6 (Change locking regimen around buffer replacement), written by Robert Haas, Amit Kapila and reviewed by Andres Freund, and commit id 3acc10c9 (Increase the number of buffer mapping partitions to 128) tested by Amit Kapila, Andres Freund and Robert Hass.
The results here are exciting. Not only have we been able to see that the Lenovo 3850 X6 system offers unparalleled performance for the Postgres Plus database, we also see that continued enhancements made in new major versions of Postgres (in this case, between 9.4 and 9.5-devel) provide meaningful improvements in the performance of the database when under scale and load.
Special thanks goes to a handful of EDBers: Jan Wieck who configured EDB Postgres Plus and ran these benchmark tests, Craig Silveira who set up and ran the hardware in his personal lab, and Robert Haas for his general guidance and knowledge.
Jason Davis is Director, Product Management, at EnterpriseDB.