We had the opportunity to sit down with Martijn Wallet, Co-Owner and CTO of OptimaData—a full-service, multi-platform data(base) service provider and our Dutch EDB Partner at KubeCon + CloudNativeCon Europe 2022. Wallet shared with us how OptimaData has leveraged Postgres database, Kubernetes and EDB’s CloudNativePG to empower their clients.
For those unable to attend the event, we’ve compiled the conversation between Wallet and EDB’s Postgres DBA and Cloud Native Lead, Gabriele Bartolini.
Hi Martijn, can you please tell us a bit about your story with OptimaData and Postgres?
Martijn Wallet: Hi Gabriele! It’s really my pleasure to be here. As a quick introduction for the audience, I am Martijn Wallet from OptimaData.
We founded OptimaData five years ago because we sensed there was a need for a multi-platform database consultancy firm. So, we created OptimaData as a full-service, multi-platform database service provider. We specialize in open source database management systems like Postgres and MySQL, but we also support SQL Server and Oracle. We deliver all services connected to database management, such as consultancy, support, maintenance, monitoring and training.
OptimaData’s database consultants are very experienced and are used to operating in complex environments. We have about 15 permanent staff and 30 that are deployed full-time with our customers. All of our staff have a wide range of knowledge and experience with multiple platforms, so no customer is ever left without an answer to their questions. We like to call our team Database Reliability Engineers, specifically because they make sure our customers run their database in the most efficient, consistent and cost effective manner.
We have all kinds of customers using all sorts of technologies—some use virtual machines (VMs) or even bare metal. Others are introducing more modern infrastructure solutions like Kubernetes into their environments.
Our goal is to serve our customers as a trusted advisor, an independent database consultancy firm invested in what’s best for them, not trying to push or sell one specific platform. To do this, we invest time in understanding every facet of our customers’ existing environment and their plans for the future. This, combined with our expertise, allows us to find the best possible solution for every client. Plus, because OptimaData handles the database concerns, our customers can focus on other challenges—which is a great relief to them.
You mentioned Kubernetes. Can you share with us why you are using Postgres and Kubernetes in production?
MW: When it comes to Kubernetes, most of the time our customers want to be able to deploy and modify applications in an easy and maintainable fashion. With the correct helm charts and tooling, Kubernetes is a very agile mechanism for deployment of applications.
Postgres is of course the most advanced open source database to choose, especially to run in containers on Kubernetes. It has a small footprint and can be extended with PostGIS and TimescaleDB.
Some of our customers are aware of the possibility to run Postgres on containers, but others are not yet ready for it. We work with the latter to help them understand the benefits through proof of concept.
With the enhancements of persistent storage in Kubernetes and the evolution of orchestrators for all kinds of databases like MySQL—with or without Galera—and Postgres, we as a consultancy firm can serve as a trusted advisor for them on these integrations.
We are here today to celebrate the launch of CloudNativePG, the open source operator originally created by EDB. We are promoting the use of databases in Kubernetes, so it is important to share success stories with the world. Would you mind sharing yours with us?
MW: Yes, Royal Van Oord. They are a dredging and marine contractor and have been a great organization to work with.
Traditionally, Van Oord was running on Crunchy Postgres containers and looking for extended Postgres Support. This was before the Crunchy operator was developed.
In early 2019, Van Oord asked OptimaData to support their Postgres containers and facilitate backup and recovery. We achieved this by using logical replication from the containers in Kubernetes to two Postgres instances on VMs.
The number of Van Oord’s Postgres containers or applications that used a Postgres pod, extended from just a handful to a few dozen. Their workloads were also different from application to application. Some are analytical, some of them are oltp. The sizes also varied quite a bit. One might be just a few GB while another could be 100 GB.
So, for Van Oord’s logical replication setup we had some challenges:
- the lifecycle of their applications (as thePostgres containers being re-initiated)
- the size of some Postgres containers and the initial data sync with logical replication (which extended the time frame)
- working with specific extensions like PostGIS and TimescaleDB which needed some attention during initialization of logical replication
All this taken into account, we advised Van Oord to use a Postgres Operator, noting it would eliminate some of those challenges and make maintenance requirements less cumbersome.
What was that decision process like for you?
MW: OptimaData created a report on the most common available operators at that time, which was the end of 2020.
At the time, these were
- The Zalando Postgres Operator
- The Crunchy Data Operator
- EDB’s Cloud Native Postgres Operator
We weighed cost, support, ease of use, flexibility, documentation and, as far as it could be determined, market share.
Why did you ultimately pick Cloud Native Postgres by EDB?
MW: When selecting an operator, it is important to look further than the current application.
With the available options, we strongly felt that, for the long term, the EDB’s Cloud Native Postgres would be the most complete and futureproof fit for this project—and potentially very interesting for Van Oord in general.
The most important factors in this decision were:
- EDB’s enterprise-grade support from EDB (and large partner network)
- The engineering quality of the operator
- Support for bi-directional replication (the only operator with this capability)
- EDB’s enterprise monitor, which provides a unified view of all the clusters and nodes, both in k8s and on IAAS
What are your top 3 capabilities for Cloud Native Postgres?
MW: Number one is definitely the very fast deployment of a cluster—setup takes minutes. And also, how little effort it takes to roll-out a pgbouncer setup—a few lines of code and there you go
For number 2: the operator causes the cluster to always appear as defined, thereby moving the cluster closer to true high availability with minimal human intervention.
Number 3: the CloudNative Postgres Plugin, cnp, which makes it very easy to manage a Postgres cluster on Kubernetes.
What major challenges have you encountered, even with running databases in Kubernetes?
MW: No database modernization comes easy. It takes hard work and collaboration with the customer.
With Royal Van Oord, there had to be a complete mindset change: from traditional deployments to a declarative model, scripts to helm charts.
Another challenge was that, at the time, there wasn’t an EDB supported image with PostGIS and TimescaleDB extensions.Of course there were the nodes, node pools, taint and affinity that could be configured. We are definitely able to experiment with and leverage that.
We did encounter some monitoring and alerting challenges, getting the right metrics into the dashboards. But the dashboards delivered with EDB’s Postgres Operator are a good starting point.
Lastly, you just have to think about resource limitation on nodes which are running pods that are more resource intensive. That’s an ever-present reality with a project like this.
Final question. What do you think of the decision from EDB to open source EDB Cloud Native Postgres and start CloudNativePG with the commitment to donate it to the CNCF?
MW: Postgres has had an incredibly successful journey over the last 20 to 30 years, which is something you would always want for an operator like EDB’s Cloud Native one. I think, when more people from the community are prepared and able to contribute, it will only make the operator stronger and more battle-tested.
I would like to take this opportunity to congratulate the community with another great product: the Cloud Native Postgres operator, now called EDB Postgres for Kubernetes. Congratulations!