Anthropic can now track the bizarre inner workings of a large language model

“On the Biology of a Large Language Model”) details what the team discovered when it looked at 10 tasks in particular.

That’s because the models are trained, not programmed.

“They almost grow organically,” says Batson. “They start out totally random. Then you train them on all this data and they go from producing gibberish to being able to speak different languages and write software and fold proteins. There are insane things that these models learn to do, but we do... See more