Huge “foundation models” are turbo-charging AI progress

The “good computer” which Graphcore, a British chip designer, intends to build over the next few years might seem to be suffering from a ludicrous case of nominal understatement. Its design calls for it to carry out 1019 calculations per second. If your laptop can do 100bn calculations a second—which is fair for an average laptop—then the Good comp... See more

To understand why foundation models represent a “phase change in ai”, in the words of Fei-Fei Li, the co-director of Stanford University’s Institute for Human-Centred ai, it helps to get a sense of how they differ from what went before.

All modern machine-learning models are based on “neural networks”—programming which mimics the ways in which brain cells interact with each other. Their parameters describe the weights of the connections between these virtual neurons, weights the models develop through trial and error as they are “trained” to respond to specific inputs with the sor... See more

Emergent properties are linked to another highly promising feature: flexibility. Earlier generations of ai systems were good for only one purpose, often a pretty specific one. The new models can be reassigned from one type of problem to another with relative ease by means of fine tuning. It is a measure of the importance of this trait that, within ... See more