Natural Intelligence
The challenges we face in understanding language models resemble those faced by biologists. Living organisms are complex systems which have been sculpted by billions of years of evolution. While the basic principles of evolution are straightforward, the biological mechanisms it produces are spectacularly intricate. Likewise, while language models a... See more
On the Biology of a Large Language Model
On the Biology of a Large Language Model
transformer-circuits.pub
That’s because the models are trained, not programmed.
“They almost grow organically,” says Batson. “They start out totally random. Then you train them on all this data and they go from producing gibberish to being able to speak different languages and write software and fold proteins. There are insane things that these models learn to do, but we do... See more
“They almost grow organically,” says Batson. “They start out totally random. Then you train them on all this data and they go from producing gibberish to being able to speak different languages and write software and fold proteins. There are insane things that these models learn to do, but we do... See more
Will Douglas Heaven • Anthropic can now track the bizarre inner workings of a large language model
“On the Biology of a Large Language Model”) details what the team discovered when it looked at 10 tasks in particular.
Will Douglas Heaven • Anthropic can now track the bizarre inner workings of a large language model
The Neubauer Collegium - The Neubauer Collegium
neubauercollegium.uchicago.edu