LLMs
sari and
LLMs
Artificial Intelligence
Timothy Shih and • 243 cards
People need to be more thoughtful building products on top of LLMs. The fact that they generate text is not the point.
thesephist.comstream.thesephist.com“A more practical answer is that it’s a file. This right here is a large language model, called Vicuna 7B. It’s a 4.2 gigabyte file on my computer. If you open the file, it’s just numbers. These things are giant binary blobs of numbers…”
Simon Willison, attempting to explain LLM
nice overview of why every company needs an AI strategy
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public