Generative AI
by Johann Van Tonder · updated 3mo ago
Generative AI
by Johann Van Tonder · updated 3mo ago
“A more practical answer is that it’s a file. This right here is a large language model, called Vicuna 7B. It’s a 4.2 gigabyte file on my computer. If you open the file, it’s just numbers. These things are giant binary blobs of numbers…”
Simon Willison, attempting to explain LLM
Johann Van Tonder added 6mo ago
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public dom
Johann Van Tonder added 6mo ago
“The fact that these things model language is probably one of the biggest discoveries in history. That you (LLM) can learn language by just predicting the next word … — that’s just shocking to me.”
- Mikhail Belkin, computer scientist at the University of California
Johann Van Tonder added 6mo ago
"A key challenge of (LLMs) is that they do not come with a manual! They come with a “Twitter influencer manual” instead, where lots of people online loudly boast about the things they can do with a very low accuracy rate, which is really frustrating..."
Simon Willison, attempting to explain LLM
Johann Van Tonder added 6mo ago
“From this point on, the intelligence of LLMs… will only continue to improve. Human intelligence will not.”
OpenAI’s paper on using AI to debug AI code
Johann Van Tonder added 3mo ago
Younger workers benefit more from labor-augmenting tech, like AI.
“In other words, workers who were used to doing things a certain way struggled to adapt when complementary technology arrived, while less-experienced workers could harness the power of these new tools.”
Source: Kellogg Insight
Johann Van Tonder added 6mo ago
“We don’t know what capabilities GPT-5 will have until we train it and test it. It might be a medium-size problem right now, but it will become a really big problem in the future as models become more powerful.”
Johann Van Tonder added 6mo ago
Skills you need in the age of AI: You’ll need critical thinking skills and emotional intelligence. Some technical skills e.g. understand the AI capabilities, some data literacy, cyber threats etc.
But mostly, you’ll need truly human soft skills:
- ability to make complex decisions
- ability to have creative problem solving
All skills that will enable
Johann Van Tonder added 6mo ago
Your "AI Chief Of Staff". Sort of assistant agent you talk to 2-3 times a day, it handles everything for you. Then go back to just working on things. "To me, that's the dream of AI and knowledge work - much more so than the AI helping me do the actual work. I don't care about the speed at which I do my tasks. I want to eliminate all the context shi
... See moreJohann Van Tonder added 6mo ago
Everyone freaks out, and they interpret the statement, ‘AI will affect my job’ as ‘AI will do my job for me.’
Those two things are not the same, because AI can be a tool or substitute. Just because the job may change doesn’t mean that the job will be eliminated.
Source: Kellogg Insight
Johann Van Tonder added 6mo ago