How bad could it be? If you ask the researchers at Anthropic, even if progress stalls out here, current algorithms will automate all white collar work within the next five years: it’s just a matter of collecting the relevant data and spoonfeeding it to the models. In the worst-case scenario, highly repetitive manual labour becomes the last frontier for human competitive advantage:

The really scary future is one in which AIs can do everything except for the physical robotic tasks, in which case you’ll have humans with airpods, and glasses, and there’ll be some robot overlord controlling the human through cameras by just telling it what to do, and having a bounding box around the thing you’re supposed to pick up. So you have human meat robots.

I doubt it happens this quickly or completely, but I think something like this happens. You can’t achieve actual AGI without bridging the creativity and agency gaps—that’s the whole point of this post—but machines can absolutely flood the market with cheap labour, not all of which will be offset by increased demand.7 The rate at which the economy changes determines whether we get a very good future or a very bad one.

Let’s say AI improves slowly enough that we get a perfectly smooth transition. Even here we get plenty of mundane problems: spam, deepfakes, AI-generated slop flooding the Internet and your inbox, fake girlfriends, highly sophisticated scams. Any person with malicious intent now has powerful tools at their disposal that give them huge leverage: maybe they can 10x or 100x their nefarious plans; maybe it’s a stepwise change in terms of brute-forcing bioweapons or something.

So by ‘mundane’ I mean actually still potentially really bad; it’s just that everything seems mundane compared to summoning an alien god that will lead you to immortal heaven/kill you and everyone you love.

AI Futures Project

Why I Am No Longer an AI Doomer

The Intelligence Age