Rebuilding Enterprise AI Around the Database

Also available on:

Ask a question in plain English and get the answer straight from your data: it’s the most intuitive promise in enterprise AI. Michael Stonebraker tested it where it counts, on queries people had actually run against MIT’s data warehouse. The LLM got none right, and even with RAG and prompting tricks it was still wrong more than four times out of five. The trouble is the data itself. It’s private, so no model has ever seen it. It’s full of local quirks, like buildings known by number rather than name. And years of cryptic, overlapping tables leave it riddled with what he calls “schema rot.”

Smarter models alone won’t fix that, and neither will putting an LLM in charge of the workflow. LLMs are strong with text and weak at combining structured data, which is most of what an enterprise holds. Stonebraker’s team built Rubicon to flip that design: LLMs only pull from text, the results become tables, and a database-style engine does the rest, at a tenth of the cost and with considerably better accuracy. Once the hype fades, he expects enterprises to be more selective, using LLMs for the text-heavy work they’re good at and database-style systems for the structured data most of their business runs on.

Key takeaways:

• Don’t trust the leaderboard. High benchmark scores say little about how AI will perform on your own data, so test it against your own warehouse before committing to it, and hold it to the 80% accuracy he says enterprises need.
• AI readiness starts in the C-suite. Accurate answers depend on data owners across the business documenting and opening up what they hold, and that rarely happens without the CEO behind it. GSK’s former data chief made read access to all of the company’s structured data a condition of taking the job.
• Trust comes before scale. Agents that can go rogue, or that answer the same question differently twice, are dealbreakers for most enterprises. By contrast, he considers open-source models run on-premises and cut off from outside networks relatively safe.
• Data-heavy agents belong inside the database. The database already has the controls it needs (access control, full logging, transaction triggers), and the security gaps sit higher up, in the application and model layers.The way to run these agents there, he argues, is as user-defined functions, which mainly need better developer tools to catch on.
• Postgres is the open-source default. Backed by many companies and controlled by none, it’s the database he recommends for most enterprises.