Why Computers Won’t Make Themselves Smarter

Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided tha... See more

This isn’t a question of proving that something is impossible; it’s a question of what constitutes good justification for belief. The critics of Anselm’s ontological argument aren’t trying to prove that there is no God; they’re just saying that Anselm’s argument doesn’t constitute a good reason to believe that God exists. Similarly, a definition of... See more

This is how recursive self-improvement takes place—not at the level of individuals but at the level of human civilization as a whole. I wouldn’t say that Isaac Newton made himself more intelligent when he invented calculus; he must have been mighty intelligent in order to invent it in the first place. 

Simple tools make it possible to create complex ones; this is just as true for cognitive tools as it is for physical ones. Humanity has developed thousands of such tools throughout history, ranging from double-entry bookkeeping to the Cartesian coördinate system. So, even though we aren’t more intelligent than we used to be, we have at our disposal... See more

For the foreseeable future, the ongoing technological explosion will be driven by humans using previously invented tools to invent new ones; there won’t be a “last invention that man need ever make.” In one respect, this is reassuring, because, contrary to Good’s claim, human intelligence will never be “left far behind.” But, in the same way that w... See more

**Note:** AI as religion - something beyond humans’ comprehension that we’re putting faith in“If the human brain were so simple that we could understand it, we would be so simple that we couldn’t.” This statement makes intuitive sense, but, more importantly, we can point to a concrete example in support of it: the microscopic roundworm C. elegans. ... See more

The rate of innovation is increasing and will continue to do so even without any machine able to design its successor. Some might call this phenomenon an intelligence explosion, but I think it’s more accurate to call it a technological explosion that includes cognitive technologies along with physical ones. Computer hardware and software are the la... See more

How much can you optimize for generality? To what extent can you simultaneously optimize a system for every possible situation, including situations never encountered before? Presumably, some improvement is possible, but the idea of an intelligence explosion implies that there is essentially no limit to the extent of optimization that can be achiev... See more

God isn’t the only being that people have tried to argue into existence. “Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever,” the mathematician Irving John Good wrote, in 1965: