Mathematics is not losing its grip on machine learning. It is changing jobs. The last decade proved that scaling compute beats elegant theory: engineering-first efforts building models with billions of parameters delivered capabilities that mathematically principled architectures could not predict or match. Rich Sutton's 'Bitter Lesson' has been swallowed again. But the response from the mathematical community is not retreat.
The article's core argument is that math's role is shifting from providing upfront theoretical guarantees to delivering post-hoc explanations of observed phenomena, the same function it performs in physics. More specifically, pure mathematical fields including topology, algebra, and geometry are now entering a space previously held by probability theory and linear algebra. The reason matters: these pure fields were built to handle abstraction and complexity at scales that break human intuition. The 40-year-old convolutional neural network, which baked translation equivariance directly into its architecture by matching data symmetry, is the prototype for where this leads. The harder problem now is that analyzing a 7-billion-parameter, 50-layer network by accuracy benchmarks alone is, as the authors put it, like describing an elephant from the touch of a single pin.
The full piece earns a read not for its conclusions but for its construction. It walks through specific research threads showing how tools from geometry and algebra are being applied to hidden activations and weight spaces, domains with millions of orthogonal directions from a single point. If you work in ML and assume math is becoming decorative, this will challenge that assumption with specifics.
[READ ORIGINAL →]