AI safetyism has become so dominant that the obsession with alignment between humans and AI could, by inhibiting accelerated progress in the field, become an existential risk in itself. There’s
AI safetyism has become so dominant that the obsession with alignment between humans and AI could, by inhibiting accelerated progress in the field, become an existential risk in itself. There’s
The tipoff to
the nature of the AI societal risk claim is its own term, “AI alignment”. Alignment with what? Human values. Whose human values? Ah, that’s where things get tricky.
the nature of the AI societal risk claim is its own term, “AI alignment”. Alignment with what? Human values. Whose human values? Ah, that’s where things get tricky.
Marc Andreessen • Why AI Will Save the World
AI alignment is just the problem of AI bias writ (terrifyingly) large: We are assigning more societal decision-making power to systems that we don’t fully understand. D o these two groups of experts charged with making AI safe actually get along? T hese are two camps, and they’re two camps that sometimes stridently dislike each other.
There are two factions working to prevent AI dangers. Here’s why they’re deeply divided.
Traditional philosophy has been going on almost three thousand years; machine goal alignment has until the advent of superintelligence, a nebulous event which may be anywhere from a decades to centuries away. If the control problem doesn’t get adequately addressed by then, we are likely to see poorly controlled superintelligences that are unintenti... See more
LessWrong • Superintelligence FAQ - LessWrong
The idea is that humans will always remain in command. Essentially, it’s about setting boundaries, limits that an AI can’t cross. And ensuring that those boundaries create provable safety all the way from the actual code to the way it interacts with other AIs—or with humans—to the motivations and incentives of the companies creating the technology.... See more
Will Douglas Heaven • DeepMind’s Cofounder: Generative AI Is Just a Phase. What’s Next Is Interactive AI.
The AI-risk community has also learned that novel corporate-governance structures cannot constrain executives who are hell-bent on acceleration. That was the big lesson of OpenAI's boardroom fiasco. “The governance model at OpenAI was supposed to prevent financial pressures from overrunning things,” Ord said. “It didn't work. The people who were me
... See moreRoss Andersen • AI Doomers Had Their Big Moment
Professionals should be the ones making the important decisions and steering AI, not letting it run loose hoping for the best.
Zach Tratar • Tweet
AI that can start to be an AI scientist and self-improvement could help us solve the hard alignment problems that we don’t know how to solve. The alignment problem is: how do we build AGI that does what is in the best interest of humanity? How do we make sure humanity gets to determine the future of humanity. It is also interesting to note how this... See more
OpenAI CEO Sam Altman | AI for the Next Era
Generally, though, consider containment more as a set of guardrails, a way to keep humanity in the driver’s seat when a technology risks causing more harm than good. Picture those guardrails operating at different levels and with different modes of implementation. In the next chapter we’ll consider what they might look like at a more granular level
... See more