Rethinking AI Forecasting: Are We Just Chasing Our Tails?

Time-series forecasting in AI has been heralded as a area where new models showcase their prowess. But color me skeptical, as a closer examination reveals a concerning trend: the reliance on datasets that favor complex architectures over efficient classical methods. What they're not telling you is that many of these benchmarks are filled with datasets rife with persistent periodicities and seasonalities that simpler models can easily handle.

The Pitfalls of Predictability

Current benchmarks often feature datasets dominated by predictable autocorrelation and seasonal cycles. These are the bread and butter for statistical models, yet they form the testing grounds for advanced deep learning architectures. The result? Complex models that offer no real advantage over their simpler counterparts. The claim that these sophisticated models significantly outperform classical methods doesn't survive scrutiny. We must ask ourselves: are the marginal improvements worth the hefty computational demands and increased complexity?

A Call for Diverse Benchmarks

Let's apply some rigor here. To genuinely advance our understanding and capabilities in time-series forecasting, AI researchers should retire or substantially augment current benchmarks. We need datasets that reflect a broader array of real-world challenges, think structural breaks, time-varying volatility, and concept drift. These elements introduce non-stationarities that are less predictable, offering a more strong test of a model's capabilities.

every deep learning submission should include classical baselines well-suited to the specific time-series characteristics being tackled. This practice would ensure that reported gains are genuinely scientific and not mere artifacts of cherry-picked datasets favoring pattern recognition.

Why It Matters

Ignoring the strengths of classical methods in favor of deep learning is akin to using a sledgehammer when a scalpel would suffice. This oversight not only wastes resources but also obscures scientific progress. We should care because the future of AI in forecasting depends on acknowledging and addressing these methodological shortcomings. It's about time the community steps up and demands more from its benchmarks to drive real progress.