AI safetyism has become so dominant that the obsession with alignment between humans and AI could, by inhibiting accelerated progress in the field, become an existential risk in itself. There’s
AI safetyism has become so dominant that the obsession with alignment between humans and AI could, by inhibiting accelerated progress in the field, become an existential risk in itself. There’s
The tipoff to
the nature of the AI societal risk claim is its own term, “AI alignment”. Alignment with what? Human values. Whose human values? Ah, that’s where things get tricky.
the nature of the AI societal risk claim is its own term, “AI alignment”. Alignment with what? Human values. Whose human values? Ah, that’s where things get tricky.
Marc Andreessen • Why AI Will Save the World
AI alignment is just the problem of AI bias writ (terrifyingly) large: We are assigning more societal decision-making power to systems that we don’t fully understand. D o these two groups of experts charged with making AI safe actually get along? T hese are two camps, and they’re two camps that sometimes stridently dislike each other.
There are two factions working to prevent AI dangers. Here’s why they’re deeply divided.
Traditional philosophy has been going on almost three thousand years; machine goal alignment has until the advent of superintelligence, a nebulous event which may be anywhere from a decades to centuries away. If the control problem doesn’t get adequately addressed by then, we are likely to see poorly controlled superintelligences that are unintenti... See more
LessWrong • Superintelligence FAQ - LessWrong
The idea is that humans will always remain in command. Essentially, it’s about setting boundaries, limits that an AI can’t cross. And ensuring that those boundaries create provable safety all the way from the actual code to the way it interacts with other AIs—or with humans—to the motivations and incentives of the companies creating the technology.... See more
Will Douglas Heaven • DeepMind’s Cofounder: Generative AI Is Just a Phase. What’s Next Is Interactive AI.
Professionals should be the ones making the important decisions and steering AI, not letting it run loose hoping for the best.
Zach Tratar • Tweet
AI that can start to be an AI scientist and self-improvement could help us solve the hard alignment problems that we don’t know how to solve. The alignment problem is: how do we build AGI that does what is in the best interest of humanity? How do we make sure humanity gets to determine the future of humanity. It is also interesting to note how this... See more
OpenAI CEO Sam Altman | AI for the Next Era
Generally, though, consider containment more as a set of guardrails, a way to keep humanity in the driver’s seat when a technology risks causing more harm than good. Picture those guardrails operating at different levels and with different modes of implementation. In the next chapter we’ll consider what they might look like at a more granular level
... See moreMustafa Suleyman • The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma
A lot of effective altruism organizations have made AI x-risk their top cause in the last few years,” explains Sayash Kapoor from Princeton. “That means a lot of the people who are getting funding to do AI research are naturally inclined, but also have been specifically selected, for their interest in reducing AI x-risk.”
Thus, the interest in AI ... See more
Thus, the interest in AI ... See more