Rethinking Solomonoff: The Quest for a Truly Universal Predictor

The concept of universal prediction has long fascinated computer scientists, with Solomonoff's approach standing as a noteworthy attempt to formalize it. At its core, Solomonoff's theory relies on the critical principle of computability. This, however, is where its troubles begin.

The Computability Conundrum

Solomonoff's framework aims to satisfy two essential computability criteria. Yet, a deeper look reveals that it falls short. The root of this issue can be linked back to a generalization of a diagonalization argument originally suggested by philosopher Hilary Putnam. By using this argument, we see the crumbling foundation of Solomonoff's universal predictor. It's a stark reminder that even the most promising theories can unravel under close scrutiny.

But why should we care? As we look at into the computational underpinnings of AI, understanding where these theories falter is as critical as knowing where they succeed. If we're building models atop shaky assumptions, the entire edifice could collapse.

Occam's Razor: A Misguided Hope?

Some advocates argue that Solomonoff's approach supports the methodological principle of Occam's razor, the idea that simpler explanations are preferable. However, if the underlying framework is flawed, does this justification hold any weight? The AI-AI Venn diagram is getting thicker, and so are the issues we must address.

Occam's razor, in its essence, isn't merely a guide but a reflection of our cognitive biases. When predicting future states or trends, we might lean toward simplicity, but is that truly prudent? Theoretical ideals are alluring, yet practicality must reign supreme.

Beyond Theoretical Ideals

Solomonoff's theory, despite its shortcomings, has influenced the development of machine learning methods. But is this influence more about idealistic aspiration than tangible application? The collision between theoretical purity and practical utility is inevitable. We're building the financial plumbing for machines, yet the foundations we rely on must be strong and realistic.

As AI systems continue to evolve, they demand frameworks grounded in reality, not mere idealism. Solomonoff's approach may inspire, but it's time to question whether inspiration alone suffices. With agentic models steering the future, we need predictors that aren't just theoretically sound but practically viable.