The case for slowing down AI

While there’s no way to uninvent the nuclear bomb or the genetic engineering tools that can juice pathogens, catastrophic AI has yet to be created, meaning it’s one type of doom we have the ability to preemptively stop.

Experts have already seen and documented  more than 60 smaller-scale examples of AI systems trying to do something other than what their designer wants  (for example, getting the high score in a video game, not by playing fairly or learning

game skills but by hacking the scoring system).

In her ideal world, we’d halt work on making AI more powerful for the next five to 10 years. In the meantime, society could get used to the very powerful systems we already have, and experts could do as much safety research on them as possible until they hit diminishing returns. Then they could make AI systems slightly more powerful, wait another