Sublime

This is from Apple's State of the Union The local model is a 3B parameter SLM that uses adapters trained for each specific feature. Diffusion model does the same thing, adapter for each style. Anything running locally or Apple's Secure Cloud is an Apple model, not OpenAI.... See more

So Apple has introduced a new system called “Private Cloud Compute” that allows your phone to offload complex (typically AI) tasks to specialized secure devices in the cloud. I’m still trying to work out what I think about this. So here’s a thread. 1/

Really fascinating discussion around Magnificent Seven, AI, and investing in AI by Gavin Baker, Antonio Gracias, and Bill Gurley. Leaving my notes here: "...what scaling laws say is if you wanna double the performance of an algorithm, you need to train it on ten more compute and data. So the reason this turbo charges... See more

There are some interesting optimizations to consider when running retrieval at scale (in @cursor_ai's case, hundreds of thousands of codebases) For example, reranking 500K tokens per query With blob-storage KV-caching and pipelining, it's possible to make this 20x cheaper (1/8)