Why Fine-Tuning Fails: The Truth About Synonym Competition in Language Models

Fine-tuning language models is a delicate art, often marred by unexpected pitfalls. One such pitfall emerges when models face near-synonym competitors in their training contexts. Despite a decrease in cross-entropy loss, the correct token fails to surpass its synonym in ranking. What's going on here?

The Experiment

This phenomenon is scrutinized across five transformer architectures, spanning two distinct families and varying parameter counts by fivefold. Researchers handpicked ten contexts highlighting near-synonym challenges, offering a rigorous testbed for this issue. The results are eye-opening: the failure to properly rank the correct token isn't just a quirk, it's a systemic issue.

What's driving this failure? The study uses an order parameter, a clever combination of the predicted distribution and embedding overlaps, to dissect the problem. This parameter splits into two components: a signal measuring model commitment to the correct token, and a background drag representing probability leakage from the embedding bulk.

Two Modes of Failure

Here’s what the benchmarks actually show: there are two distinct failure modes. In 'kinematic failure', the signal remains weak, suggesting the model never truly commits to the right answer. In 'structural failure', the background drag exacerbates as fine-tuning progresses, actively dragging down performance.

Interestingly, sharp shifts in the order parameter resemble phase transitions, but these are misleading. The so-called transitions are phantoms, debunked by direct measurements. Even when employing LoRA fine-tuning, where the token embedding matrix remains constant, these jumps persist. The discontinuity is confined to the softmax readout.

A Predictive Framework

One standout finding is the organization of model trajectories through a series of dimensionless quantities. These quantities remain consistent across all five architectures under full fine-tuning. One of them even predicts LoRA sufficiency and sorts architectures into two classes based on bulk embedding distribution.

Here's a bold prediction: this framework accurately forecasts the critical learning rate for an entirely new architecture, hitting the mark within 2.1% during subsequent learning-rate sweeps. This isn't just technical jargon. It points to a deeper, often overlooked aspect of model training, one that could redefine how we approach fine-tuning.

Why should this matter to you? Strip away the marketing and you get a clearer picture: our understanding of model training is far from complete. These failure modes reveal unseen complexities that could influence how effective models are in real-world applications. And if you're deploying AI in any capacity, isn't it time to question how reliable your models are when they face similar challenges?

In the end, this study invites us to rethink how we fine-tune models, especially in contexts where synonyms battle for supremacy. The reality is, successful AI deployment demands more than just lowering loss metrics. It requires a nuanced understanding of the hidden dynamics at play.