Data & Architecture
A Shift in the Foundation of the Datastore
The decision of which database to choose for a new enterprise application project (the ‘persistence’ model) is the most influential and irreversible choice a software architect can make. In the early days of the internet, relational databases (RDBMS), such as PostgreSQL and MySQL, ruled the world with an iron fist. They offered ACID guarantees (Atomic, Consistent, Isolated, Durable), which meant that your data was written reliably and in a structured manner. However, over the past few decades, the growth of cloud-scale applications has exploded, resulting in massive, complex architectures and asynchronous data flows. This introduced unexpected bottlenecks into traditional relational database architectures.
Relational databases have traditionally been designed to scale ‘vertically’ (scale up): if you need more capacity, you must upgrade the physical server (more RAM and CPU). This vertical scalability has a hard, physical, and enormously expensive limit. This created the perfect storm for the NoSQL revolution, and today, the rise of NewSQL systems.
The NoSQL Movement (MongoDB and Cassandra)
To meet the demands of web-scale applications, the NoSQL (Not Only SQL) movement emerged in the early 2010s. Databases such as MongoDB (Document-based) and Cassandra (Wide-Column Store) rejected rigid table structure and foreign keys. Their primary advantage: they are designed to scale horizontally (scale-out) on commodity hardware (cheap cloud servers). If your database fills up, you simply add a new server (‘node’) to the cluster, and the database automatically distributes (shards) the data.
Developers appreciated NoSQL, specifically document stores like MongoDB, because they could send JSON objects directly from their application (JavaScript/NodeJS) to the database without the complex translations of an Object-Relational Mapper (ORM). However, they traded transactional guarantees (ACID) and powerful ad-hoc join queries for this, which often proved to make NoSQL unsuitable for critical financial or e-commerce domains where data consistency is the number one priority.
The Renaissance of PostgreSQL
Meanwhile, the relational ‘old guard’ has not stood still. PostgreSQL is praised by developers today as the most advanced open-source database in the world. It has adopted many features that made NoSQL unique. With the native JSONB With this datatype, you can store and index unstructured JSON documents in PostgreSQL and execute lightning-fast queries on them. As a result, PostgreSQL functions as a capable ‘NoSQL’ document store for many organizations, while still retaining the absolute power of relational consistency, triggers, constraints, and subqueries.
NewSQL: Best of Both Worlds (CockroachDB & Spanner)
What if you need the absolute ACID transactional guarantees and SQL language of PostgreSQL, but also the horizontal, distributed scalability and robustness (resilience against downtime) of Cassandra? This need created the newest category in the database world: ‘NewSQL’ (or Distributed SQL).
Technologies such as Google Cloud Spanner and open-source alternatives like CockroachDB and TiDB are leading the way in this new field. CockroachDB, whose name refers to the indestructibility of the cockroach, is designed to survive a complete data center fire or network outage without data loss or downtime. It distributes data blocks across multiple data centers or continents and uses distributed consensus algorithms (such as Raft) to ensure that nodes are always in agreement on the status of transactions, regardless of network latency. The fact that developers communicate with these systems via a standard, trusted SQL connection significantly lowers the barrier to entry. Read in-depth about the connection between databases and cloud architectures on the experts hub of Computable.
Next:
