The experiment of social media was: build a virtual room, let everybody in, reward the people who scream that loudest, and then see what happens. The experiment of AI seems to be: build an interlocutor with specific ideology and élan, scale it infinitely across the Internet, and see what happens.
What is the social cost of scaling a technology to millions of people who trust it for the most important life questions, if that technology lacks the critical insight to help a user who doesn’t know how to help themselves?
he study “Towards Understanding Sycophancy in Language Models” (published 2023; updated 2025) found that both human evaluators and preference models can “prefer convincingly-written sycophantic responses over correct ones.” In other words, the more chatbots are designed to appeal to people, the more they specialize in telling people exactly what... See more