Optimizer Research Progress
For the original motivation, approach, and initial results, refer back to Experimental Optimizer Research & Progress. This update focuses on the progress made since that write-up.
Worklog Posts
My worklog posts and overall technical jargon posts. Will be writing about mathematics, AI, and computer science projects.
-
-
Optimizer Research & Progress
Muon, a SOTA-level optimizer, takes a new approach toward optimization. By analyzing and hypothesizing how its mechanics work, one can try to design a similar optimizer from the same core ideas. Since Muon orthogonalizes 2D update matrices…