Overfitting in ML Benchmarks: The Illusion of Complexity

Machine learning's benchmark obsession has long had skeptics crying overfitting. Yet, oddly enough, reality seems to disagree. Benchmark-driven ML has skirted the overfitting trap, leaving many scratching their heads. So, what's the secret sauce? One theory: ML strategies are shockingly compressible.

The Compression Hypothesis

Picture this. You've got LLM-driven research agents. These agents, tasked with finding the best models, do so with efficiency that'd make a Swiss watch jealous. The trick? Two forms of information bottlenecks: output and input compression.

Output compression tests whether a simple, short prompt paired with training data can reproduce the performance of high-performing models. On the other hand, input compression gives feedback in one-bit increments, signaling whether a new model outdoes the current leader. Across eight datasets, ranging from tabular classification to reward modeling, these methods have proven surprisingly effective. High performance, minimal complexity.

The Elephant in the Room

But let's not get ahead of ourselves. The hypothesis isn't bulletproof. Inducing overfitting on a validation set throws the whole model out the window. No short prompts can save a model drowning in validation-set overfitting. It's a falsifiable scenario, and it fails spectacularly.

So, what's the takeaway? Successful ML strategies might just reside in a low-complexity neighborhood. They're like tenants in a rent-controlled building, thriving under conditions that'd buckle others. It's a description-length explanation for the lack of overfitting. But should we trust it?

Why Simplicity Might Win

Does this mean complexity is overvalued? Maybe. If simple strategies are beating the odds, it's high time we reassess our obsession with complexity. We've seen it before, everyone has a plan until the model collapses under its own weight.

So, ask yourself. Are we overthinking our benchmarks? With machines that thrive on simplicity, maybe it's time to strip away the excess. Zoom out. No, further. See it now?