Saved by Andrés and
The Illustrated Transformer
The challenges we face in understanding language models resemble those faced by biologists. Living organisms are complex systems which have been sculpted by billions of years of evolution. While the basic principles of evolution are straightforward, the biological mechanisms it produces are spectacularly intricate. Likewise, while language models a... See more
On the Biology of a Large Language Model
Imagine an LLM as a diligent apprentice chef who aspires to become a master chef. To learn the culinary arts, the apprentice starts by reading and studying a vast collection of recipes from around the world. Each recipe represents a piece of text, with various ingredients symbolizing words and phrases. The apprentice’s goal is to understand how to
... See moreEthan Mollick • Co-Intelligence: Living and Working with AI
transformers instead introduce the notion of “attention”—and the idea of “paying attention” more to some parts of the sequence than others.
Stephen Wolfram • What Is ChatGPT Doing ... And Why Does It Work?
The Transformer solved these issues by utilizing an “attention mechanism.” This technique allows the AI to concentrate on the most relevant parts of a text, making it easier for the AI to understand and work with language in a way that seemed more human.