There are two factions working to prevent AI dangers. Here’s why they’re deeply divided.

Of course, there are major ethical issues to work out—leaps forward in technology often walk a fine line between deeply-impactful and dystopian. Among the questions we need to figure out:

• Who is responsible for AI’s mistakes?


• Who is the creator of an AI work? Is it the AI? The developers? The person who wrote the prompt? The people whose work was use

... See more

Economists have observed

a longstanding pattern in reform movements of this kind. The actors within movements like these fall into two categories – “Baptists” and “Bootleggers” – drawing on the historical example of the prohibition of alcohol in the United States in the 1920’s:

Baptists

“Baptists” are the true believer social reformers who legitimate... See more

Outside of film, people tell credible dystopian AI narratives around entrenching wokeism more deeply, or worse, replacing human work. AI can be a symbiotic partner to human creativity and thought, but the optimistic story needs to be shared.

AI’s positive mission is not as crystallized as crypto’s, or at least not yet.

Unclear fo

... See more

A third reason to worry about the alignment problem of computers is that because they are so different from us, when we make the mistake of giving them a misaligned goal, they are less likely to notice it or request clarification. If the boat-race AI had been a human gamer, it would have realized that the loophole it found in the game’s rules proba

... See more

OpenAI’s Misaligned Superaligners > they had undeniably good intentions > but companies, economies, nations, are built on compromise > leadership is forced to make those decisions sometimes Counterfactual > Jan and Superalignment team were listened to > 4o release was delayed to 2025 > what would have happened? > Elevenlabs had professional q... See more

Professionals should be the ones making the important decisions and steering AI, not letting it run loose hoping for the best.

Traditional philosophy has been going on almost three thousand years; machine goal alignment has until the advent of superintelligence, a nebulous event which may be anywhere from a decades to centuries away. If the control problem doesn’t get adequately addressed by then, we are likely to see poorly controlled superintelligences that are unintenti... See more

The idea is that humans will always remain in command. Essentially, it’s about setting boundaries, limits that an AI can’t cross. And ensuring that those boundaries create provable safety all the way from the actual code to the way it interacts with other AIs—or with humans—to the motivations and incentives of the companies creating the technology.... See more