Sublime

Unlike humans, whose brains must accompany their bodies, the memory system of an intelligent machine might be located remotely from its sensors (and “body,” if it had one).

As Turing concluded, they need an “oracle”—a source of intelligence outside the system itself—and all he could say about the oracle is that it “could not be a machine.” Turing saw that computers repeat the uncertainties of physics that stem from recursive self-reference.

Friston believed that minimizing prediction error—roughly the same as minimizing free energy—caused the body to act. True, this account of bodily action sounded a bit peculiar. How does the brain cause an arm to move? It predicts that the arm is moving. Proprioceptive sensors issue frantic error signals to the muscle telling it that the arm is not

The big question, as we evolve into the future of the hyperintelligent machine, is not a question of control but one of ethics.

He concludes that there is a common function, a common algorithm, that is performed by all the cortical regions.

This need for specialization, the constrained bandwidth of our ability to communicate, our limited memory capacity and processing power, means even the smartest of minds is approaching the limits of human intelligence.

it boils down to searching, among myriad possible settings of the internal model, for those that best correspond to the state of the external world.