Beware the Intention Economy: Collection and Commodification of Intent via Large Language Models

identifies this type of personal AI ‘assistant’ as the equivalent of a “market avatar” that steers conversation in the service of platforms, advertisers, businesses, and other third parties.

In an intention economy, an LLM could, at low cost, leverage a user’s cadence, politics, vocabulary, age, gender, preferences for sycophancy, and so on, in concert with brokered bids, to maximize the likelihood of achieving a given aim (e.g., to sell a film ticket).