Decoding Neural Forecasting: Horizon Activation Mapping's New Vision

In the area of time series forecasting, neural networks have long relied on error metrics and architecture-centric interpretability methods that often fail when models differ in family. Enter Horizon Activation Mapping (HAM), a fresh visual interpretability technique that's shaking up the landscape with a nod to the grad-CAM method used in image data analysis. Instead of focusing solely on attention maps, HAM leverages gradient norm averages to dig into into the subseries of the forecasting horizon.

Visualizing the Future

HAM isn't just another tool in the AI toolkit. It's a strategic approach to interpret models without being tied down to specific layer types or architectures. By introducing causal and anti-causal modes, HAM calculates gradient update norm averages across subseries at every timestep, offering a uniform distribution perspective. The AI-AI Venn diagram is getting thicker.

But why does this matter? For starters, it enhances our understanding of how different model families behave across training, validation, and test sets. By studying optimization landscapes, including factors like batch sizes, early stopping, and train-val-test splits, HAM provides a granular view of model performance.

A New Lens on Performance

What makes HAM intriguing is its application across multivariate forecasting models, such as MLP-based CycleNet, N-Linear, N-HITS, and self-attention-based models like FEDformer and Pyraformer. Even SSM-based SpaceTime and diffusion-based Multi-Resolution DDPM models trained on the ETTm2 dataset are dissected through HAM plots.

The results reveal some unexpected insights. Differences in activities tied to batch sizes suggest a potential exponential approximation across epochs, offering a new angle on optimization. Moreover, NHITS' neural approximation theorem and SpaceTime's exponential autoregressive activities manifest as distinct trends in HAM plots. This isn't a partnership announcement. It's a convergence.

Why Care About HAM?

Model selection and validation are essential, and HAM provides a lens to see beyond the surface. Isn't it time we ask: if agents have wallets, who holds the keys? HAM answers part of this by enabling comparisons across neural network families, aiding in precise model selection.

In a field obsessed with accuracy and efficiency, tools like HAM offer a much-needed layer of transparency and understanding. As AI continues to evolve, the convergence of different technologies and methodologies will dictate the pace of progress. We're building the financial plumbing for machines, and HAM represents a vital cog in that machinery. The compute layer needs a payment rail.