Building software
sari and
Building software
Pros
- The query speeds improve as, instead of instructing Elasticsearch to Get all the data matching... , you say Go there and get all the data matching... ( Fig 6 ). If you are querying data with multiple unique routing keys, this guarantees better or equivalent query performance as that of the default routing strategy (reducing the scatter and
Improving query performance in ElasticSearch at scale
Cloudflare serves around 20% of the web with 46 million requests a second.
Surely they must have a lot of data.
Where do they store it?
Plain old PostgreSQL. 🐘
Around 15-20 clusters of them.
Each cluster... See more
I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. It demands the same skill, devotion, insight, and even inspiration as the... See more
I’ve built software that iterates slower than SpaceX is iterating starships.
Between 1 & 2: new actuation, launch pad cooling, hot staging and on and on
“For any given technology development, it’s how many iterations you have and the time between... See more
Patrick OShaughnessyx.comdegree of iteration in software development is inevitable. As soon as one accepts that logic, one must then also accept that software development will be subject to friction (i.e. unexpected inefficiencies). When I plan the future for a piece of software, I expect to encounter ongoing friction, and I try to allow for that. I remain astonished at how many organisations assume that a master plan for their software can be rigidly followed.
- The number of partitions is fixed at the time of the index creation. Each partition is an indivisible chunk of data which can grow in size but not be split into smaller chunks. As the data grows, more nodes may be added to the cluster, but the number of partitions which eventually move to the newer nodes are limited (Fig 3)
- In some sense, as the
Improving query performance in ElasticSearch at scale
The choice of the routing key in our case was a combination of the unique identifier (UUID) and type of the primary participant of each event in a calendar. This choice was made based on the querying patterns of the consuming application and to maintain a degree of uniqueness.
