LLMs
sari and
LLMs
Using LLM products today feels a lot like using early cars in the 1800s: clearly magical, clearly going to change the world, and really hard to drive.
The first cars didn’t have steering wheels (they hadn’t been invented yet), so you’d steer them with a big lever called a tiller. The problem with tillers is that they are imprecise, which made... See more
The first cars didn’t have steering wheels (they hadn’t been invented yet), so you’d steer them with a big lever called a tiller. The problem with tillers is that they are imprecise, which made... See more
“A more practical answer is that it’s a file. This right here is a large language model, called Vicuna 7B. It’s a 4.2 gigabyte file on my computer. If you open the file, it’s just numbers. These things are giant binary blobs of numbers…”
Simon Willison, attempting to explain LLM
"A key challenge of (LLMs) is that they do not come with a manual! They come with a “Twitter influencer manual” instead, where lots of people online loudly boast about the things they can do with a very low accuracy rate, which is really frustrating..."
Simon Willison, attempting to explain LLM
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public
There is so much more structure in the world than we tend to believe. And LLMs and other AI models are structure discovery machines.
Albert Wengerx.com
1/ AI exploded on the scene at the end of 2022, with applications like @OpenAI's ChatGPT taking the world by storm.
2023 is poised to be the most exciting year of AI yet! 🎉
Here's mine and @vivek7ue's look at the lay of the land + what might happen next...
👀🧵... See more