Sublime

Nvidia literally said small LLMs are the future of AI agents https://t.co/w5jlRvF5Kh

The <100B Model Red Wedding I do not think that people who criticize OpenAI have sufficiently absorbed the magnitude of disruption that has just happened because of 4o mini Llama 3 70b: 82 MMLU, $0.90/mtok gpt 4o mini: 82 MMLU, $0.15/mtok every model on the RHS side of this chart is now strictly dominated by their LHS counterparts some of these models were SOTA 3 months ago. what is the depreciation rate on the FLOPs it took to train them? gpt4 took $500m to train and it lasted ~a year. intelligence too cheap to meter, but also too ephemeral to support >5 players doing R&D? is there an angle here i'm missing? the other angle i have been thinking a lot about is the separation of reasoning from knowledge. RAG/memory plugs knowledge easily but not reasoning. 82 MMLU is plenty. you can get it up to 90, but its not going to be appreciably smarter in normal use without... See more

Q*? 2015: reinforcement learning prompt engineer in Sec. 5.3 of “Learning to Think...” https://t.co/5FQEb7Cc3F. A controller neural network C learns to send prompt sequences into a world model M (e.g., a foundation model) trained on, say, videos of actors. C also learns to interpret answers of M, extracting algorithmic information from... See more

The reason humans are so useful is not mainly their raw intelligence. It’s their ability to build up context, interrogate their own failures, and pick up small improvements and efficiencies as they practice a task.

Mike Krieger is Instagram co-creator and Anthropic CPO. He's predicting AI's next trillion-dollar wave. It's not better models. The real fortune will come from something much simpler. His roadmap for the coming AI gold rush: https://t.co/VJShBvikk6