Superintelligence: Paths, Dangers, Strategies
So far, we have defined a learning rule. To get an agent, we also need a decision rule. To this end, we endow the agent with a “utility function
Nick Bostrom • Superintelligence: Paths, Dangers, Strategies
The stages in this sequence correspond to whole brain emulations of successively more neurologically sophisticated model organisms—for example, C. elegans → honeybee → mouse → rhesus monkey → human. Because the gaps between these rungs—at least after the first step—are mostly quantitative in nature and due mainly (though not entirely) to the
... See moreNick Bostrom • Superintelligence: Paths, Dangers, Strategies
To assess the feasibility of whole brain emulation, one must understand the criterion for success. The aim is not to create a brain simulation so detailed and accurate that one could use it to predict exactly what would have happened in the original brain if it had been subjected to a particular sequence of stimuli. Instead, the aim is to capture
... See moreNick Bostrom • Superintelligence: Paths, Dangers, Strategies
Historically, AI researchers have not had a strong record of being able to predict the rate of advances in their own field or the shape that such advances would take. On the one hand, some tasks, like chess playing, turned out to be achievable by means of surprisingly simple programs; and naysayers who claimed that machines would “never” be able to
... See moreNick Bostrom • Superintelligence: Paths, Dangers, Strategies
In the third stage, the neurocomputational structure resulting from the previous step is implemented on a sufficiently powerful computer. If completely successful, the result would be a digital reproduction of the original intellect, with memory and personality intact.
Nick Bostrom • Superintelligence: Paths, Dangers, Strategies
For example, many types of artificial neural network can be viewed as classifiers that perform a particular kind of statistical calculation (maximum likelihood
Nick Bostrom • Superintelligence: Paths, Dangers, Strategies
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the
... See moreNick Bostrom • Superintelligence: Paths, Dangers, Strategies
happiness. (Similar issues would arise if we the programmer were interested in justice, freedom, glory, human rights, democracy, ecological balance, or self-development.)
Nick Bostrom • Superintelligence: Paths, Dangers, Strategies
Nils Nilsson has spent a long and productive career working on problems in search, planning, knowledge representation, and robotics; he has authored textbooks in artificial intelligence; and he recently completed the most comprehensive history of the field written to