At each stage in this “training” the weights in the network are progressively adjusted—and we see that eventually we get a network that successfully reproduces the function we want. So how do we adjust the weights? The basic idea is at each stage to see “how far away we are” from getting the function we want—and then to update the weights in such a

guaranteed in “deeper” networks with more than two layers, it was possible to build a system that could produce results that were often good enough, opportunistically climbing up the mountain by taking small steps of the right sort, using a technique called backpropagation—now the workhorse of deep learning.*4 Backpropagation works by estimating th

Provided you can learn them, networks with many layers can express many functions more compactly than SVMs, which always have just one layer, and this can make all the difference.