AI - Ongoing Model Improvements

One way of thinking about this is Daniel Kahneman’s simple model of thinking: System 1 and System 2. System 1 thinking is fast and intuitive. Current AI models’ pattern recognition and next-token prediction are good examples of this. System 2 thinking is slow and analytical, akin to genuine reasoning and understanding. It is System 2 thinking where... See more

This is all early, and based on first impressions, but I think that voice capabilities like GPT-4o’s are going to change how most people interact with AI systems. Voice and visual interactions are more natural than text and will have broader appeal to a wider audience. The future will involve talking to AI.

LeCun points to four essential characteristics of human intelligence that current AI systems, including LLMs, can’t replicate: reasoning, planning, persistent memory, and understanding the physical world. He stresses that LLMs’ reliance on textual data severely limits their understanding of reality: “We’re easily fooled into thinking they are intel... See more

I isn’t ready to be an entrepreneur, but its ideation-prototype-interview cycle does in a couple of seconds what takes my students months to do. AI isn’t ready to build educational games without errors, but it is able to instantly make an interactive simulation that explains a difficult concept, even if some nuance is missing

He (Claude Shannon) had this great intuition that information is maximized when you’re most surprised about learning about something.” ~ Tara Javidi

“Whenever we are surprised by something, even if we admit that we made a mistake, we say, ‘Oh I’ll never make that mistake again.’ But, in fact, what you should learn when you make a mistake because you... See more

The incentive to find breakthrough science that provides a performance pathway other than scaling is increasing. A GPT-6 class model will cost $10 billion and three years to improve on GPT-5 by some uncertain degree. That’s a ton of time and a lot of cash for an uncertain payout: in other words, a substantial prize for anyone who can figure out pro... See more

As Ethan said a year ago, it takes 10 hours of monkeying around to get into it. But I think a good AI executive training program can dramatically shorten the window, assuming a growth mindset on the exec’s part. I ran into a retired architect the other day who was bemoaning wanting to work in retirement but he had never learned CAD-CAM! Now he’s tw... See more

While LLMs continue to devour web-scraped data, they’ll increasingly consume their own digital progeny as AI-generated content continues to flood the internet. This recursive loop, experimentally confirmed, erodes the true data landscape. Rare events vanish first. Models churn out likely sequences from the original pool while injecting their own un... See more

Arbituram

Jul 4

Same here; I'm not rubbishing the benchmarks, they have their uses, but Sonnet 3.5 just *feels* so far ahead of anything else, I had to go for a long walk on the beach after using it to contemplate how long humanity has left.

Sonnet 3.5 is able to have thoughtful and nuanced conversations on complex topics and to be proactive in codin... See more