Huge “foundation models” are turbo-charging AI progress
economist.com
Saved by Sterling Proffer
Huge “foundation models” are turbo-charging AI progress
This moment has
been decades in the making. Six decades of Moore’s Law have given us the compute horsepower to process exaflops of data. Four decades of the internet (accelerated by COVID) have given us trillions of tokens’ worth of training data. Two decades of mobile and cloud computing have given every human a supercomputer in the palm of our han... See more
been decades in the making. Six decades of Moore’s Law have given us the compute horsepower to process exaflops of data. Four decades of the internet (accelerated by COVID) have given us trillions of tokens’ worth of training data. Two decades of mobile and cloud computing have given every human a supercomputer in the palm of our han... See more
Sonya Huang • Generative AI’s Act Two
Parker C added
How did we get to the doorstep of the next leap in prosperity?
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “ru... See more
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “ru... See more
samaltman.com • The Intelligence Age
How did we get to the doorstep of the next leap in prosperity?
In three words: deep learning worked.
In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it.
That’s really it; humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data). To a shocking degree of precision, the more compute and data available, the better it gets at helping people solve hard problems. I find that no matter how much time I spend thinking about this, I can never really internalize how consequential it is.
sari and added
A rough analogy to the current LLM process is that making a new model is like baking a cake. You figure out your data and algorithms—like mixing the batter—and then you pretrain the model, that is, run it on a large number of computers for several months—like putting it in the oven—and then at the end you do some “post training”—like frosting and d... See more
Avital Balwit • My Last Five Years of Work
Max Beauroyre added
Model managers of tomorrow will need to learn the same things. They’ll need to know which AI models to use for which tasks. They’ll need to be able to quickly evaluate new models that they’ve never used before to determine if they’re good enough. They’ll need to know how to break up complex tasks between different models suited to each piece of wor... See more
Dan Shipper • The Knowledge Economy Is Over. Welcome to the Allocation Economy
Britt Gage added
