FilterMoE: The Next Step in Graph Neural Network Evolution?

Graph neural networks (GNNs) have taken a significant leap forward with the introduction of pre-propagation techniques. These approaches push graph-dependent computation to a preprocessing phase, using dense hop features for actual training. This leads to greater scalability. However, a fascinating puzzle has emerged: simple MLP-based aggregators often match or surpass their more complex hop-attention counterparts. What's going on here?

Revisiting Graph Filters

The key difference lies in how filter coefficients are shared across nodes and feature channels, not just the raw capacity of the aggregators. MLP-based architectures tend to learn channel-dependent filters that are largely node-independent. In contrast, hop-attention architectures focus on node-dependent mixtures, shared across channels. This reveals a gap in current PPGNN models: they're missing joint node- and channel-adaptive filtering.

Introducing FilterMoE

Enter FilterMoE, a novel mixture-of-experts PPGNN. It employs a small bank of Chebyshev filter experts, which a 3D gating tensor routes jointly over nodes and channels. The result? Across eleven competitive benchmarks, FilterMoE outperformed leading PPGNN baselines on nine datasets. It even clinched the top spot on all three large-scale tests, boosting the average test score by 1.53 points. That's no small feat.

Beyond Dataset-Specific Solutions

FilterMoE's success brings a provocative question to the fore. Are we over-engineering dataset-specific solutions when a more balanced, adaptive approach like FilterMoE might offer superior results? It suggests a shift in how we design these systems, moving towards joint node-channel filter routing rather than sticking to the tried-and-tested hop-aggregator selection.

This isn't just about technical superiority. It's a convergence of efficiency and scalability that could redefine how we approach GNNs. The AI-AI Venn diagram is getting thicker, and FilterMoE is a potent symbol of that evolving intersection.