How open source Postgres is meeting the data needs of the future
Twenty eight years ago, EDB VP Bruce Momjian chose to work on Postgres. That was a defining moment in his life, not to mention a fortunate career choice. Because unlike many of the databases and operating systems that Bruce used in the beginning of his career, Postgres is still here, still growing, and is taking over the rest of the relational database space.
In a recent presentation, Bruce shared insights on the evolution and advantages of PostgreSQL. He also provided career advice that applies to not just Postgers developers, but everyone working in technology.
Here are the highlights of the discussion:
Why is Postgres so successful?
One reason Postgres has changed the world so dramatically and advanced so quickly is because of its open source development process. Unlike other relational databases, Postgres is developed by community volunteers around the world, working 24 hours a day in various time zones. This is how we’ve been able to release major versions of Postgres every year since 1986.
Open source can be developed inside a company, as Elasticsearch, MongoDB, and Redis have done, where the company releases the code but the developers are hired internally. But Postgres is different because developers from different companies are effectively collaborating to come up with the best solutions.
While Postgres continues to grow, proprietary databases like Oracle, Microsoft SQL and IBM Db2 are declining in comparison. Many of these companies support Postgres. Even though IBM has Db2, they still support Postgres, and so does Microsoft SQL Server and Amazon Web Services (AWS). Oracle has even created its own version of Postgres.
Postgres is highly extensible
Another reason Postgres has not just survived but thrived, is because of its extendability. This extensibility has allowed Postgres to meet the data needs of the future.
Our industry started with disk drives and moved on to punch cards, teletypes, dumb terminals, and fat and thin clients, until things started to change in the 2000s with web browsers connecting to databases. Then in the 2010s, the landscape was transformed with the introduction of sensor-based data, geolocation data, loaded documents, mobile apps, GIS documents, social media, and more. This is why there are so many databases out there now, each built to store different types of data..
A lot of this new data didn’t fit into the traditional table with column structure. But because Postgres was built to be extensible, it’s easy to add specialized capabilities for the dynamic data layout that didn’t exist before.
Postgres merges relational capabilities with non-relational ones, which allows it to do almost anything. “This is one of the reasons why Postgres has become so popular,” says Bruce. “Not because we’re better than Oracle at relational, but because we’re able to do non-relational things as well as relational things.”
Postgres’ vibrant extension ecosystem makes it easy to add new data. You can load GIS data types and data analysis in Postgres, you can add operators and more procedure languages, and you can do SQL. We’ve built in full-text search and data warehousing and JSON capabilities. For cases where you can’t use Postgres, there’s a foreign data wrapper that allows you to access data that resides on other databases. This flexibility is key to Postgres’ popularity.
Focus on foundational technology and dive deep
One of the things Bruce does as an engineer is to identify technologies that he thinks are foundational to the future. “I learn the internals and dive deep into those. And by doing that I kind of future proof myself,” he says. “I’m always learning; I’m always trying to understand where the market is going.”
As a consultant for a law firm in the 90s, Bruce focused on Unix, because he saw it as a foundational technology. His colleague specialized in Microsoft Windows config.sys memory management and WordPerfect macros. As the WordPerfect word processor declined and Windows changed immersion, his colleague had to switch gears, because his work was tied to a specific piece of software rather than a foundational technology
One suggestion Bruce has for developers is to find technology you really feel is foundational and then take the time to go beyond the surface level of the technology. Doing the deep research needed to understand the full stack or answer questions is key. “If you invest to get to the expertise level, you can solve questions really quickly,” he says.
Proprietary software vs open source development
In Bruce’s experience, companies that attempt to take Postgres and make it a million times better usually fail after a year or two. “Unless you find a way to synchronize your development with the Postgres community so you can get the benefits of the community every year, then you’re going to fall behind, almost by definition,” he says.
The main reason proprietary solutions can’t compete against Postgres is because proprietary software development is incredibly inefficient. With proprietary software, all the developers are in the company and the users are separate. Users only see the features once they get to the release. But with open source, users are involved during the entire development process.
“It’s not that Postgres people are brilliant,” he says. “It’s the process that’s brilliant. And that’s the fundamental reason you keep seeing more and more open source and why companies that develop proprietary software have trouble.”
Key career advice: Pursue your passion
In the PostgreSQL community, contributors gravitate towards areas that spark their interest, A few new Postgres community members were motivated to work on optimizer capabilities – and you’ll find these in the upcoming Postgres 17 release. Others are infrastructure engineers who want to focus on the build system, APis and testing. There are people want to fix bugs, and others who want to work on big features.
If you’re an engineer interested in working on Postgres, you have to decide if you’re an infrastructure person, or if bug fixes motivate you, or if you want to work in a green field and effectively develop a feature from start to finish. With Postgres, people work on what they’re interested in and find rewarding.
Postgres isn’t just a database. It’s a way of life.
Bruce shared a story about a Russian nuclear research student who emailed his doctoral presentation to the Postgres community. In his dissertation, he fixed a complex issue in PostgreSQL. The community reached out and invited the student to improve that area of Postgres.
While there wasn’t an immediate benefit to doing the work, this student agreed to collaborate with the Postgres community. His contribution was quickly integrated into a PostgreSQL release, leading to speaking invitations at conferences worldwide.
This student’s success captured the interest of his university, who invited him and Bruce to speak there, too. Nine months later, two Russian developers approached Bruce at a San Francisco conference and shared that they were there because they were so inspired by the university talk. Today, these two developers are an integral part of the Postgres community.
“If you’re part of the 5% who are really excited about open source and what can be done, that would be a match for the community,” says Bruce.” It’s only going to be a tiny number of people. That’s ok. We have a small group of crazy people, and we have three conferences a month, and other crazy people come and we talk about databases. And we enjoy doing it.”
If you’re in the 5% we encourage you to get involved. You can explore the PostgeSQL community, learn how to become a contributor and find out about upcoming conferences on the Postgres community site.