Possible Minds: Twenty-Five Ways of Looking at AI

inventing, for example, imaginary video games and painting screen shots from them.

It’s that the biggest threats lie in the networks of ideas, norms, and institutions that allow information to feed back (or not) on collective decisions and understanding.

Though one can imagine hypothetical combinations of the most malevolent totalitarians with the most advanced technology, in the real world it’s the norms and laws we should be vigilant about, not the tech.

the reward is meant to incentivize robot behavior that matches what the end user wants, what the designer wants, or what society as a whole wants.

Whatever actions the robot decides on need to mesh well with ours. This is the coordination problem.

If information was like entropy, then it could not be conserved—or contained.

formulation, the information content of a string of symbols was given by the logarithm of the number of possible symbols from which a given string was chosen.

For Shannon, “information” could be quantified because its fundamental unit, the bit, was a unit of conveyance rather than understanding.

let it pull out the salient features. Instead, Lake et al. gave the program a general model of how you draw a character: A stroke goes either right or left; after you finish one, you start another; and so on. When the program saw a particular character, it could infer the sequence of strokes that were most likely to have led to it—just as I inferre

We are not stuff that abides, but patterns that perpetuate themselves.*