Saved by sari and
AI Revolution - Transformers and Large Language Models (LLMs)
In sum, returns on intelligence will compound exponentially. A select few artificial intelligences that we used to call organizations will massively benefit from a new concentration of ability—probably the greatest such concentration yet seen. Re-creating the essence of what’s made our species so successful into tools that can be reused and reappli
... See moreMustafa Suleyman • The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma
On the Biology of a Large Language Model
transformer-circuits.pub
Talking about large language models
arxiv.orgIn 2020, Jared Kaplan and his collaborators at OpenAI suggested that there was a set of “scaling laws” for neural network models of language; they found that the more data they fed into their neural networks, the better those networks performed.10 The implication was that we could do better and better AI if we gather more data and apply deep learni... See more
Gary Marcus • Deep Learning Is Hitting a Wall
ModelsText is the most advanced domain. However, natural language is hard to get right, and quality matters. Today, the models are decently good at generic short/medium-form writing (but even so, they are typically used for iteration or first drafts). Over time, as the models get better, we should expect to see higher quality outputs, longer-form c... See more
Pat Grady • Generative AI: A Creative New World
