🧠 AI’s $100bn question: The scaling ceiling

Indeed, we may already be running into scaling limits in deep learning, perhaps already approaching a point of diminishing returns. In the last several months, research from DeepMind and elsewhere on models even larger than GPT-3 have shown that scaling starts to falter on some measures, such as toxicity, truthfulness, reasoning, and common sense

In 2023, AI must absorb an influx of tourists – thin GPT wrappers, MBA tweet threads about AI trends, LinkedIn bios changing from #crypto to #AI. Genuine technologists will join the development effort, but filtering out negative human capital is challenging.

Undoubtedly, more builders are needed to apply and productionize the latest technologies,

... See more

I still think that we are thinking about scaling of network-based intelligence looking at this particular network. An extended network of machines and humans in larger networks facilitated by smaller and increasingly embodied devices might in themselves add an AI-augmented neocortex to our (human-machine) systems. Intelligence will continue to emer... See more

As the LLM market structure stabilizes, the next frontier is now emerging. The focus is shifting to the development and scaling of the reasoning layer, where “System 2” thinking takes precedence. Inspired by models like AlphaGo, this layer aims to endow AI systems with deliberate reasoning, problem-solving and cognitive operations at inference time... See more