Sublime

Doing Matrix Calculus can be messy, specially when we need higher order derivatives. Writing them out using Tensor Diagrams makes even the Hessian Chain Rule relatively simple:

time to go down the rabbit hole ig https://t.co/PUMGbMiRXp

This is a great high-level introduction to Hamiltonian Monte Carlo methods that I found very accessible. It has great visual explanations of a variety of relevant concepts. https://t.co/0YI8OYHDNR

I've finally solved steepest descent on Finsler-structured (matrix) manifolds more generally. This generalizes work by me, @jxbz, and @Jianlin_S on Muon, Orthogonal Muon, & Stiefel Muon. --- The general solution turned out to be much simpler than I thought. And it should generalize to any... See more

Hash3D Training-free Acceleration for 3D Generation The evolution of 3D generative modeling has been notably propelled by the adoption of 2D diffusion models. Despite this progress, the cumbersome optimization process per se presents a critical hurdle to efficiency. In this... See more

wrote a paper: it lets you *train* in 1.58b! could use 97% less energy, 90% less weight memory. leads to a new model format which can store a 175B model in ~20mb. also, no backprop! https://t.co/RwBheY9wej

Natural gradient with momentum on submanifolds: In Riemannian normal coordinate systems, metric tensor becomes identity but computationally intractable. We introduce generalized normal coordinates https://t.co/D82DD38hQz