The edge of stability, in your browser
I built a small gradient-descent visualizer. The loss is a
stretched bowl, L = ½(x² + 3y²). Nothing fancy, but it makes one thing obvious.
Because the y direction is 3× steeper, the largest stable learning rate is set by that
direction, not by the average curvature. Push the rate past roughly 0.66 and y oscillates
with growing amplitude while x is still crawling toward zero. That gap, one axis diverging
while another has barely moved, is the whole reason preconditioning and adaptive methods exist.
Open the demo, drag the start point out to (-4, 3), and sweep the learning rate up from 0.1.
Watch exactly where the path flips from a smooth curve, to a zig-zag, to a blow-up. Seeing it
beats reading the inequality.