Will scaling reasoning models like o1, o3 and R1 unlock superhuman reasoning? I asked Gwern + former OpenAI/DeepMind researchers. Warning: long post. As we scale up training and inference compute for reasoning models, will they show: A) Strong... See more

something obviously true to me that nobody believes: 90% of frontier ai research is already on arxiv, x, or company blog posts. q* is just STaR search is just GoT/MCTS continuous learning is clever graph retrieval +1 oom efficiency gains in... See more

# A new type of information theory this paper is not super well-known but has changed my opinion of how deep learning works more than almost anything else it says that we should measure the amount of information available in some representation based on how *extractable* it is, given finite... See more