Introducing CloudNativePG: A New Open Source Kubernetes Operator for Postgres

Today is the culmination of years of hard work at EDB, and, hopefully, the beginning of a new phase in the multi-decade evolution of Postgres and its community.

On April 21, 2022, EDB released CloudNativePG—an open source Kubernetes operator that manages highly available primary/standby cluster architectures for the Postgres database management system.

CloudNativePG is distributed under the Apache License 2.0, and is now owned and governed by a newly formed community of contributors to the project, built on solid principles and values inspired by the Cloud Native Computing Foundation (CNCF). Among these are openness, fairness, inclusivity, technical excellence, “community over product/company,” built-in quality and built-in security. I’m part of the initial group of maintainers of the project, currently made up by the top six committers of the project within EDB.

CloudNativePG is a commitment to Postgres and Kubernetes

Our adoption of this open and vendor-neutral model was influenced by our past experiences and the vision we have for the future of Postgres in Kubernetes. It reflects our direct contributions to the Postgres community for the past two decades, as well as our multi-year commitment to making CloudNativePG a graduated project within the Cloud Native Computing Foundation (CNCF).

There are currently three maturity levels or stages defined for CNCF projects: sandbox, incubation, and graduation. In order to move to the next stage, down through final graduation, each project needs to prove that it’s credible, sustainable, widely adopted, has a healthy rate of changes, and is developed by contributors from multiple organizations. 

In the words of AC/DC, “It’s a long way to the top.” However, if given the opportunity by the Technical Oversight Committee (TOC) of the CNCF to enter the Sandbox, we are committed to leading CloudNativePG through the entire journey, hopefully with participation from users and vendors who are equally invested in improving the database experience with Postgres in Kubernetes.

We believe that a CNCF-endorsed open source operator will help more users run Postgres database workloads inside Kubernetes, alongside their client applications. These workloads will be perfectly integrated with the other components of their infrastructure, including monitoring, alerting, logging, tracing, storage, security and compliance. Moreover, running Postgres workloads in the same Kubernetes cluster in which your applications run enables the organizational transformation that is typical of DevOps. 

It’s no coincidence that EDB’s been the first Postgres company to become a Kubernetes Certified Service Provider (KCSP), as well as to be directly involved in the Data on Kubernetes (DoK) Community as a founding Sponsor to push the boundaries of data workloads in Cloud Native environments - specifically Postgres databases.

The story of CloudNativePG

CloudNativePG was originally conceived in August 2019, when I was asked by Simon Riggs, then CEO at 2ndQuadrant now EDB, to lead the Cloud Native/Kubernetes initiatives for the company. My team had a long running history of DevOps practices. We had been following Kubernetes for some time—how can I forget the “aha!” moment when our friends at Zalando showed us how Postgres was running in their production Kubernetes, for the first time!

The game changer for us was the introduction of local persistent volumes in Kubernetes 1.14 (April 2019), together with a more consistent and standard adoption of the operator pattern in the industry. We began an exploratory phase with a fail fast principle in 3 areas in parallel: achievement of the KCSP status, prototyping of an operator and benchmarking shared nothing database architectures on bare metal Kubernetes. We required the operator to rely on light images (immutable application containers), to be entirely declarative and to be tightly integrated with the Kubernetes API server (this also implied avoiding use of an external tool for failover like our repmgr, or Patroni, or Stolon, to name a few).

By December 2019, we had achieved results well beyond our expectations in all areas—most notably that Kubernetes on bare metal was performing almost identically to Linux on bare metal. If it had failed, our commitment likely would not have been so strong (see this blog from around that time). 

Next, we started the productization phase of a BDR operator for multi-master replication, which was released in January 2020 (and is being re-released later this year). In February 2020, we began refactoring that code into a new product: Cloud Native Postgres (this is our “Initial import” commit). The first failover implementation was yet another “aha!” moment that confirmed the unprecedented potential for high availability we could reach with our Postgres + Kubernetes combination.

After 2ndQuadrant was acquired by EDB in September 2020, Cloud Native Postgres was further developed, with a first stable release announced in February 2021. Since then, the product has been successfully deployed in production by EDB and other large enterprises. In fact, EDB based the data layer of BigAnimal on Cloud Native Postgres.

During the first year of Cloud Native Postgres, we became more conscious of the accelerated adoption of Kubernetes for production database workloads, and received a growing number of requests from customers with a mandate for open source solutions in their standardized infrastructure. These two factors, combined with EDB’s core goal to run Postgres everywhere for everyone, led us to the decision to submit Cloud Native Postgres to the CNCF Sandbox  in Q2/2022. 

On April 21, we renamed Cloud Native Postgres into CloudNativePG, open sourced the repository with a history of over 1400 commits, released version 1.15.0, and submitted the project to the CNCF. 

Postgres + Kubernetes: for the people

CloudNativePG ensures that everyone can start using an entirely open source stack for Postgres in Kubernetes for all their applications, with all the freedom they demand. This model empowers students, developers and start-up enterprises. It’s the culmination of our vision for Postgres.

EDB is here to help

If your organization requires professional assistance on Postgres in Kubernetes with CloudNativePG, you can take advantage of EDB’s leading expertise in both Postgres and Kubernetes technologies.

EDB offers commercial support on both the open source CloudNativePG (under the Community360 break-and-fix subscription plan), as well as on our new commercial offering “EDB Postgres for Kubernetes,” based on the EDB Cloud Native Postgres operator and available under Standard and Enterprise subscription plans. 

EDB Postgres for Kubernetes provides a certified operator for OpenShift 4.6+, RedHat UBI-based Postgres images, long term support on the operator and access to EDB Postgres Advanced under the Enterprise plan. 

Reflecting on the journey

This is an epic moment, both for me and for the trusted team of fantastic human beings I’ve worked with all these years. 

Exactly 10 years ago, my team released BARMAN as an open source project for managing backup and recovery of Postgres databases. Now, I’ve been lucky again to push the “public visibility” button for another project. The innovative DevOps mindset that has always pushed us out of our comfort zone, constantly encouraging us to reject inward thinking and, instead, reach out to our community and database sector, fishing for new and better ideas. This is exactly what the great CNCF ecosystem represents for us and for Postgres today.

I will be speaking about Kubernetes in Postgres at the Data on Kubernetes Day before KubeCon Europe 2022, and at the EDB booth. Come and stop by to chat with us and to find ways to contribute to the project! If you're not visiting Kubecon this year, join EDB's live virtual event from Kubecon on Tuesday May 17 from 3:30 pm to 5:00 pm CET to watch a preview of how Postgres runs inside the Kubernetes cluster, including an installation, configuration and deployment demo. 

For more information about Kubernetes and CNCF, watch the Honeypot Project’s comprehensive two-part documentary. If you’d like to learn more about EDB’s CloudNativePG, read our documentation, here.

 

About the author

A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After contributing to the growth of 2ndQuadrant and its members through nurturing a lean DevOps culture, he is now leading the Cloud Native initiative at EDB.