The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

Principal-component analysis (PCA), as this process is known, is one of the key tools in the scientist’s toolkit. You could say PCA is to unsupervised learning what linear regression is to the supervised variety.

The solution is to marry machine learning with game theory,

Metalearning is remarkably successful, but it’s not a very deep way to combine models. It’s also expensive,

Our goal is to figure out the simplest program we can write such that it will continue to write itself by reading data, without limit, until it knows everything there is to know.

no learner is immune to the curse of dimensionality. It’s the second worst problem in machine learning, after overfitting.

Bagging generates random variations of the training set by resampling, applies the same learner to each one, and combines

in. Instead, nearest-neighbor takes a shortcut: if the image in its database most similar to the one Jane just uploaded is of a face, then so is Jane’s. For this to work, the database needs to contain an image that’s similar enough to the new one—for example, a face with similar pose, lighting, and so on—so the bigger the database, the better.

But machine learning will cause more elections to be close in the future. In politics, as in everything, learning is an arms race.

Minsky’s theory of intelligence, as expressed in his book The Society of Mind, could be unkindly characterized as “the mind is just one damn thing after another.” The Society of Mind is a laundry list of hundreds of separate ideas, each with its own vignette. The problem with this approach to AI is that it doesn’t work; it’s stamp collecting by com