Generative AI
"A key challenge of (LLMs) is that they do not come with a manual! They come with a “Twitter influencer manual” instead, where lots of people online loudly boast about the things they can do with a very low accuracy rate, which is really frustrating..."
Simon Willison, attempting to explain LLM
I've been asked by few first year PhD about how to start LLM research on X, say long context modeling. My number one suggestion -- though it seems a bit of unconventional -- is *not* to read any papers related to long-context, but to talk to the model
- Talk to the model about a text book, course slides, financial... See more
Yao Fux.com“The fact that these things model language is probably one of the biggest discoveries in history. That you (LLM) can learn language by just predicting the next word … — that’s just shocking to me.”
- Mikhail Belkin, computer scientist at the University of California
In a situation like this, there are two general pieces of advice one can give: keep your options open, and pay close attention to what's happening. Then if you can't predict the changes AI will cause, you'll at least be able to react quickly to them.
Paul Grahamx.com4 questions of organizations [about using AI].
- What did you do that was valuable that's no longer valuable?
- What impossible things can you now do that you could not before?
- What can you democratize and bring down market?
- What can you do upmarket so you have new ways of competing?
Ethan Mollick, https://www.forbes.com/sites/jenamcgregor/2024/
Even if LLMs don’t get smarter (though I suspect they will, and soon) new capabilities and modes of interacting with AIs, like agents and massive context windows, will help LLMs do dramatic new feats.
Ethan Mollick • Which AI should I use? Superpowers and the State of Play
Ethan Mollick
you haven’t really experienced AI until you have had three sleepless nights of existential anxiety, after which you can start to be productive again
Ethan Mollick • Which AI should I use? Superpowers and the State of Play
Meta AI released LLaMA ... and they included a paper which described exactly what it was trained on. It was 5TB of data.
2/3 of it was from Common Crawl. It had content from GitHub, Wikipedia, ArXiv, StackExchange and something called “Books”.
What’s Books? 4.5% of the training data was books. Part of this was Project Gutenberg, which is public
A few places you should probably use AI to help you, based on the research we have (but do whatever you want):
-Idea generation
-Writing (not writing for you, maybe, but second opinions/proofing/etc)
-Advice/finding frameworks to solve problems
-Summarizing documents & meetings
Ethan Mollickx.com