LLMs
sari and
LLMs
I've learned so much from @hubermanlab, but podcasts make recall and discovery challenging.
Huberman AI uses the latest models from @openai to search every episode with Google-level accuracy and summarize the results.
https://t.co/1nctiAIuGO https://t.co/VpjnY2NcpM
riley tomasektwitter.com“A more practical answer is that it’s a file. This right here is a large language model, called Vicuna 7B. It’s a 4.2 gigabyte file on my computer. If you open the file, it’s just numbers. These things are giant binary blobs of numbers…”
Simon Willison, attempting to explain LLM
Darren LI and
Weird GPT token for Reddit user davidjl123, “a keen member of the /r/counting subreddit. He’s posted incremented numbers there well over 163,000 times. Presumably that subreddit ended up in the training data used to create the tokenizer used by GPT-2, and since that particular username showed up hundreds of thousands of times it ended up getting
... See moreMeta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public
There is so much more structure in the world than we tend to believe. And LLMs and other AI models are structure discovery machines.
Albert Wengerx.com