Both pursuing and not pursuing new technologies is, from here, fraught with risk.

What if we could distill the essence of what makes us humans so productive and capable into software, into an algorithm?

attempting to ban development of new technologies is itself a risk: technologically stagnant societies are historically unstable and prone to collapse. Eventually, they lose the capacity to solve problems, to progress.

We need to follow the chain of reasoning to its logical end point, without fear of where that might lead,

Given the increasing availability of the tools, the presenter painted a harrowing vision: Someone could soon create novel pathogens far more transmissible and lethal than anything found in nature. These synthetic pathogens could evade known countermeasures, spread asymptomatically, or have built-in resistance to treatments. If needed, someone could

AI introduced a host of threats requiring proactive responses. It might lead to massive invasions of privacy or ignite a misinformation apocalypse. It might be weaponized, creating a lethal suite of new cyberweapons, introducing new vulnerabilities into our networked world.

We’d just had a day of talking about the end of the world, but there was still pizza to eat,

alongside these benefits, AI, synthetic biology, and other advanced forms of technology produce tail risks on a deeply concerning scale.

Technologies can fail in the mundane sense of not working: the engine doesn’t start; the bridge falls down. But they can also fail in a wider sense. If technology damages human lives, or produces societies filled with harm, or renders them ungovernable because we empower a chaotic long tail of bad (or unintentionally dangerous) actors—if, in the ag