Time Series Forecasting: Balancing Accuracy and Realism

Multi-step time series forecasting isn't just about nailing down numbers. It's about understanding the balance between accuracy and realism. Traditional metrics like mean squared error (MSE) focus heavily on point accuracy. But is that enough?

The Conditional Uncertainty Gap

Visualize this: you're forecasting multiple steps ahead. With every step, uncertainty grows. The conditional mean, often our target, starts to stray from typical realized values. This is the conditional uncertainty gap at work. It's a gap that reveals a fundamental trade-off in forecasting, between hitting precision and capturing the true variability of future events.

Why does this matter? Because whenever there's a nonzero gap, no deterministic model can both minimize MSE and accurately reflect the marginal distribution of future outcomes. Essentially, models face a choice: point accuracy or marginal realism. And this choice isn't just academic. It reshapes how we evaluate predictive models.

Empirical Findings and Practical Implications

Numbers in context: across nine real-world forecasting benchmarks, the data shows a clear pattern. A minor relaxation in MSE, by just 5%, can lead to substantial gains in marginal realism. Median improvements hit 17.3%, with some datasets seeing gains over 30%. The trend is clearer when you see it. As forecasting horizons expand, so does the complexity of the attainable accuracy-realism frontier, forming a pronounced Pareto front.

What does this Pareto front signify? It's simple. It separates MSE-optimal predictors, often under-dispersed, from those that trade a bit of accuracy for a more realistic depiction of variability. Direct multi-output strategies lean towards accuracy. Meanwhile, recursive strategies and sample-based inference favor realism.

A Structural Shift in Forecasting

One chart, one takeaway: the reliance on MSE as the sole metric in model evaluation is flawed, especially for long-horizon forecasting. This structural failure mode underscores the need to rethink strategy and inference selection through the lens of the accuracy-realism trade-off.

Why should this matter to forecasters? Because it challenges the status quo. Is it better to be precisely wrong or approximately right? The answer isn't black and white, but the implications for decision-making, policy planning, and risk management are immense.

Ultimately, the forecast landscape is evolving. As conditional uncertainty rises, so should our approach to evaluating and selecting predictive models. The chart tells the story: balance isn't just beneficial. It's essential.