Benchmarking is hard. Benchmarking databases is even harder. Benchmarking databases that follow different approaches (relational vs document) is harder still. But the market demands these kinds of comparisons. Despite the different data models that MongoDB and PostgreSQL expose, many organizations face the challenge of picking either technology. And performance is often the main deciding factor.
This is why EDB partnered with Alvaro Hernandez, the Founder of OnGres, to evaluate the performance benchmarks comparing PostgreSQL 11.1 and MongoDB 4.0. His team worked over a 5-month period to evaluate the performance of the two systems using three benchmarks:
-
Transaction processing
-
Online analytical processing (OLAP)
-
Online transaction processing (OLTP)
Horses ready! What was faster, MongoDB or PostgreSQL?
You can find the detailed results in a nearly 50 page white paper available here, but here are a few of the highlights of what he discovered.
Transaction Processing Compared: PostgreSQL 11.1 and MongoDB 4.0
Given the recent addition of transaction capabilities to MongoDB, it wasn’t too surprising to see a win for Postgres in this one, but the magnitude of the difference was still impressive. The Postgres database management system (DBMS) measured between 4 and 15 times faster than MongoDB in transaction performance testing.
Winner: PostgreSQL 11.1
Online Analytical Processing (OLAP) Compared: PostgreSQL 11.1 and MongoDB 4.0
Interestingly, Postgres demonstrated a performance advantage in a JSON-based online analytical processing (OLAP) test designed specifically to focus on document-based data. This was a surprising finding, given this is MongoDB’s presumed strength!
Winner: PostgreSQL 11.1
Online Transaction Processing (OLTP) Compared: PostgreSQL 11.1 and MongoDB 4.0
Additional testing was conducted on online transaction processing (OLTP) workloads. For the OLTP test, the industry standard sysbench benchmark was used with Postgres once again coming out on top, performing three times faster than MongoDB on average.
Winner: PostgreSQL 11.1
Additionally, across all benchmark types, OnGres found that as the datasets becomes bigger than the available memory capacity, the Postgres performance advantage grows over MongoDB.
Want to Put it to the Test yourself?
The benchmark was designed to be reproducible and run on a public cloud, so anyone who wants to compare Postgres and MongoDB can easily do so. All the code used in testing is open source and available on GitLab, so you can independently reproduce and audit these results as well. OnGres is sharing all the information on the tests they ran, why they were selected and the results they found so that anyone can reproduce the results or change parameters and configurations for their own needs.
If this was a horse race, Postgres would win by a mile. The tests show that If you want a database that delivers great performance and low latency, you want Postgres.
The testing methodology and results are available here for download.