LLMs
sari and
LLMs
Language models can take a big chunk of text and smush it down like a foot crushing a can of Coke. Except it doesn’t come out crushed—it comes out as a perfectly packaged and proportional mini-Coke. And it’s even drinkable! This is a Willy Wonka-esque magic trick, without the Oompa Loompas.
I've learned so much from @hubermanlab, but podcasts make recall and discovery challenging.
Huberman AI uses the latest models from @openai to search every episode with Google-level accuracy and summarize the results.
https://t.co/1nctiAIuGO https://t.co/VpjnY2NcpM
riley tomasektwitter.comLanguage models need auteurs and creative directors, not “thumbs up if you liked this personality”
Nabeel S. Qureshix.compersonality as the moat
“A more practical answer is that it’s a file. This right here is a large language model, called Vicuna 7B. It’s a 4.2 gigabyte file on my computer. If you open the file, it’s just numbers. These things are giant binary blobs of numbers…”
Simon Willison, attempting to explain LLM
NFX’s Generative Tech Open Source Market Map
docs.google.comMeta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public
One way to think about (LLM) is that about 3 years ago, aliens landed on Earth. They handed over a USB stick and then disappeared. Since then we’ve been poking the thing they gave us with a stick, trying to figure out what it does and how it works.
