AI - Ongoing Model Improvements

This is all early, and based on first impressions, but I think that voice capabilities like GPT-4o’s are going to change how most people interact with AI systems. Voice and visual interactions are more natural than text and will have broader appeal to a wider audience. The future will involve talking to AI.

One way of thinking about this is Daniel Kahneman’s simple model of thinking: System 1 and System 2. System 1 thinking is fast and intuitive. Current AI models’ pattern recognition and next-token prediction are good examples of this. System 2 thinking is slow and analytical, akin to genuine reasoning and understanding. It is System 2 thinking where... See more

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are... See more

Mickey Schafer

Jul 4

This past semester, a student of mine despaired over analyzing a 20+ question survey with 92 responses. She uploaded the spreadsheet to NotebookLM, a tool we'd used in class, which not only cheerfully assured her it would do the task, but also returned basic R values with short statements about strong relationships. She was... See more

First, operationalisation takes a long time, often revealing hidden potential in existing technologies. As Jack Clark notes, “if we stopped all AI progress today, there’s a huge capability overhang”. Even without further model development, building the right scaffolding can unlock surprising capabilities. This scaffolding isn’t just software; it... See more

The incentive to find breakthrough science that provides a performance pathway other than scaling is increasing. A GPT-6 class model will cost $10 billion and three years to improve on GPT-5 by some uncertain degree. That’s a ton of time and a lot of cash for an uncertain payout: in other words, a substantial prize for anyone who can figure out... See more

I still think that we are thinking about scaling of network-based intelligence looking at this particular network. An extended network of machines and humans in larger networks facilitated by smaller and increasingly embodied devices might in themselves add an AI-augmented neocortex to our (human-machine) systems. Intelligence will continue to... See more

As Ethan said a year ago, it takes 10 hours of monkeying around to get into it. But I think a good AI executive training program can dramatically shorten the window, assuming a growth mindset on the exec’s part. I ran into a retired architect the other day who was bemoaning wanting to work in retirement but he had never learned CAD-CAM! Now he’s... See more

t doesn’t help that the two most impressive implementations of AI for real work - Claude’s artifacts and ChatGPT’s Code Interpreter - are often hidden and opaque