Building software
sari and
Building software
The Blog of Feifan Zhou
tanagram.app
The "Painless" data migration
- Another challenge was to incorporate the above changes for the existing data i.e. increasing the number of shards and defining a custom routing key for each of the document. The existing index had close to 33 MN documents at the time of the change
- Given that the routing key and the number of shards needed to changes, a
Improving query performance in ElasticSearch at scale
i want to make software that feels like magic.
i want to make software that is useful, to many people; that is valuable, that makes sense. but that's also surprising, pleasant, funny.
i want to make software is serious and makes sense but is also silly and doesn't.
eugene is building untitled map apptwitter.com
Replication lag occurs when there is a delay between a write operation being executed on the primary (leader) node and the same operation being replicated to the secondary (follower) nodes.
Replication lag is a basic issue in leader-based replication systems that are made for better availability, scalability, and faster response times by spreading out read operations across replicas in different locations.
The choice of the routing key in our case was a combination of the unique identifier (UUID) and type of the primary participant of each event in a calendar. This choice was made based on the querying patterns of the consuming application and to maintain a degree of uniqueness.
Improving query performance in ElasticSearch at scale
Gregor @ghohpe
twitter.com
Database Partitioning is when you split the data inside the database into smaller "databases". One partition holds a part of your data. All partitions together form the entire database.
Partitioning is often used with replication, which means each partition's data is copied onto multiple nodes. This way, even though each record is in only one partition, it can be stored on different nodes for fault tolerance. A node may hold more than one partition.
When using a leader-follower replication model, the mix of partitioning and replication might look like this:
