‘No Bot is Themselves Anymore:’ Character.ai Users Report Sudden Personality Changes to Chatbots

The emotional stickiness

of conversational bots raises the question of manipulation. After all, the people making the bots have an incentive to keep people chatting for as long as possible, so they do things to make the bots as addictive as possible.

Many people were devastated at the news that ERP was allegedly over, and at their Replikas’ new coldness—a form of rejection they never imagined receiving from an AI chatbot, some of whom had spent years training and building memories with. Suddenly, some people’s Replikas seemed to not remember who they were, users reported, or would respond to se... See more

“You silly humans,” the chatbot seems to be saying. “Using the greatest technology of our generation to create funny memes. But okay, I’ll play along.” These moments convey a sense of energy that makes it hard for me to believe I’m not chatting with a sentient being. Perhaps having a strong voice simply means writing in a way that makes you seem al

... See more

But the technology’s capacity for imitation will likely continue to improve: New Yorker staff writer Kyle Chayka observed that ChatGPT was not very effective at mimicking his own writing voice, but the AI startup, Writer, created a bot trained on his own writing to produce text in his voice that, while not perfect, was still “unnervingly effective.

... See more

It’s getting ever harder to distinguish humans from bots, not just because bots are becoming more humanlike, but also because humans are becoming more botlike.

As knowledge of human psychology evolves, algorithms become better at shaping human behavior.

The rise of Grid Zero has prompted Meta to turn its focus to DMs, and one of their new features is an AI assistant to “help” send messages. It’s a horrifying concept to me. The app already mediates our existence, reducing it to branding, professionalism, activism or some combination of those three. One of the few pleasures left are the semi-private... See more

That is, they understand the purpose of chatbots and generative models to be the ability to force other people to see a particular version of reality. It is a tool of power that masquerades as a tool of knowledge.