LLMs
sari and
LLMs
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public
There is so much more structure in the world than we tend to believe. And LLMs and other AI models are structure discovery machines.
Albert Wengerx.comNFX’s Generative Tech Open Source Market Map
docs.google.com
March 2023 AI Overview for Boards
docs.google.comnice overview of why every company needs an AI strategy
An Interview With Daniel Gross and Nat Friedman about the Democratization of AI
Ben Thompsonstratechery.com
People are intrigued by LLMs’ ability to generate new text, change styles, and generally chat. While impressive, turning text written in one style into text written in your own voice (or into legaleze) is essentially taking unstructured data and turning it into more unstructured data. I’m surprised about the lack of attention to doing the exact... See more
Matt Hartman • The power of AI is its ability to turn unstructured data into structured data
Language models need auteurs and creative directors, not “thumbs up if you liked this personality”
Nabeel S. Qureshix.compersonality as the moat