Building software

degree of iteration in software development is inevitable. As soon as one accepts that logic, one must then also accept that software development will be subject to friction (i.e. unexpected inefficiencies). When I plan the future for a piece of software, I expect to encounter ongoing friction, and I try to allow for that. I remain astonished at how many organisations assume that a master plan for their software can be rigidly followed.

Database Partitioning is when you split the data inside the database into smaller "databases". One partition holds a part of your data. All partitions together form the entire database.

Partitioning is often used with replication, which means each partition's data is copied onto multiple nodes. This way, even though each record is in only one partition, it can be stored on different nodes for fault tolerance. A node may hold more than one partition.

When using a leader-follower replication model, the mix of partitioning and replication might look like this: