Rethinking Forecasting: The Power of Simplicity

In the area of probabilistic forecasting, simplicity often triumphs in unexpected ways. The ConformalNaive interval, a basic forecasting method, is challenging the status quo by outperforming more complex models in one-step-ahead online forecasting.

Breaking Down the Numbers

Consider the numbers: Across 2,217 real series from nine public datasets including Monash and METR-LA, ConformalNaive outperformed traditional naive value-quantile baselines and even the Conformal Seasonal Pools (CSP) method. Specifically, it surpassed the NPTS family in 73% of cases, the SeasonalNPTS in 64%, and the CSP method in 71% of series. The chart tells the story. It's a classic case of 'less is more.'

In a statistical showdown, ConformalNaive holds its ground against simpler learned conformal predictors like RCI and quantile regression. It's only when pitted against adaptive and ensemble methods such as SPCI and AgACI that it lags, trailing by a 9-33% margin in relative Winkler scores. However, this 'naive' method edges out trained neural forecasters like DeepNPTS calibration accuracy, covering the truth 84-85% of the time at a nominal 95% compared to DeepNPTS's 66%.

Why Should We Care?

So, why should this matter? In a world that often equates complexity with progress, ConformalNaive serves as a reminder that sometimes simple methods yield powerful results. One chart, one takeaway: simplicity shouldn't be underestimated.

When multi-step seasonal horizons come into play, the dynamics shift. The random-walk floor, once a strong contender, turns into the weakest link, while the seasonal pool method (CSP) takes the lead. It’s a fascinating boundary. The trend is clearer when you see it mapped out.

The Case for Simplicity

This brings us to ConformalNaive+. This one-line, training-free method adapts to the horizon and chooses the better of two complementary floors, restoring coverage. It's a straightforward yet effective approach that's hard to ignore.

The question arises: Why aren't more researchers using this as a baseline when claiming gains from learned probabilistic forecasters? Insisting on ConformalNaive as a mandatory baseline could illuminate the true value of claimed advancements.

Visualize this: a world where simplicity is the starting point. It might not only make easier research but also highlight true innovation. Numbers in context: sometimes the simplest solution isn’t just a baseline, it’s a benchmark.