Large Language Models (LLMs)
sari and
Large Language Models (LLMs)
NFX’s Generative Tech Open Source Market Map
docs.google.comThere is so much more structure in the world than we tend to believe. And LLMs and other AI models are structure discovery machines.
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public dom
Meanwhile, LLM results are probabilistic. 10 blue links are also probabilistic, but Google is giving you options - adding human curation by making you pick from a list of results. Using an LLM result is more like hitting ‘I’m feeling lucky’ - it requires more confidence
Darren LI and
“The fact that these things model language is probably one of the biggest discoveries in history. That you (LLM) can learn language by just predicting the next word … — that’s just shocking to me.”
- Mikhail Belkin, computer scientist at the University of California
