Stability and scalability for search

blog.x.com

RelatedInsightsHighlights

In case anyone is interested, here's how I deploy things: Marketing: Astro (mostly static, vanilla JS is fine). Deployed to Cloudflare Pages. - Blog, Docs, Pricing, SEO Pages. - tRPC for server-to-server calls during... See more

Shayan x.com

Scalability is crucial - systems need to be designed with the assumption that query volume, document corpus size, indexing complexity etc. could increase by 10x. What works at one scale may completely break at a higher scale.

Sharding the index, either by document or by word, is important to distribute the indexing and querying load across machines.

Claude

Twitter System Architecture

InterviewNoodle medium.com