Mirror Descent's Hidden Sensitivity: Why It Matters

Mirror Descent (MD), a method extending beyond the familiar Euclidean geometry, has gained traction machine learning for its role in reinforcement learning and post-training of large language models. However, this tool comes with an unexpected twist: it's surprisingly sensitive to its initial conditions.

The Sensitivity Puzzle

While Gradient Descent (GD) and its quadratic-regularized counterparts thrive in stability, MD reveals a starkly different picture under non-quadratic regularizers. It's like comparing a calm sea to a whirlpool just below the surface. The stability that GD enjoys doesn't always translate to MD, even when both start from a well-conditioned base.

Consider a scenario with a convex, smooth objective and a regularizer that's both strongly convex and smooth. In this setup, a tiny initial perturbation, let's call it epsilon, can snowball into something much bigger with repeated MD iterations. We're talking exponential growth here, something GD sidesteps entirely.

Why Should We Care?

Picture this: an algorithm supposed to fine-tune a model, instead spiraling out of control due to a small misstep in initialization. It's the kind of risk that should have tech leaders across industries asking tough questions. Can businesses relying on machine learning afford such volatility?

In high-dimensional or near-boundary scenarios, even linear objectives can turbocharge this sensitivity. It's like building a house of cards on a windy day.

Solutions and Trade-offs

One potential fix, as the research suggests, is adding a Bregman regularization term that anchors the MD process. But here's the kicker: the anchor's choice is make-or-break. Anchor it at the initialization, and you might only patch the problem. But anchor it at a stable fixed point, and you've got a shot at a more reliable approach.

As the AI industry rushes to keep up with constant innovation, this hidden sensitivity is a reminder. Automation isn't neutral. It has winners and losers. The productivity gains went somewhere. Not to wages. In the quest for efficiency, who's paying the cost?