The first two concerns I have above are technical in nature. In other words, something happened to the system that it wasn’t designed for. A company got hacked. An attacker emulated a real user and got access to their DA without authorization.

But the one that’s even more scary is when that doesn’t happen, and things work exactly as they’re supposed

... See more

Capturing a comprehensive dossier of users’ lives may help AI become the perfect assistant, but it also gives these systems all the necessary tools for deep manipulation —redirecting our deepest longings into buying a product, say, or joining a political movement.

“We caution that AI tools are already being developed to elicit, infer, collect, record, understand, forecast, and ultimately manipulate and commodify human plans and purposes,” he said.