Rethinking GNN Design Rules: Lessons from Facebook's Data

graph neural networks (GNNs), the industry often assumes that design principles extend uniformly across datasets. However, emerging evidence suggests otherwise. The performance of these networks, particularly the choice of aggregator functions, appears to diverge significantly when applied to different data sources.

Graph Aggregator Variability

Recent studies have focused on 24 node-classification datasets that cut across various graph types, from citation networks to more socially complex structures like Facebook's dense friendship networks. Typically, GNNs use aggregators such as sum, mean, and max to perform optimally across benchmarks. But here's the twist: the expected predictive power of edge homophily falls short when determining performance gaps between GIN-Sum and GIN-Mean.

Facebook-100, a dataset characterized by dense social connections, challenges this notion. In these graphs, label informativeness is nearly zero, yet sum aggregation shows a marked improvement, enhancing performance by 7-10% and even reaching 13% with extended training. This isn't simply a factor of mean degree, as stochastic block model ablations, even those adjusted to match Facebook-100's degree scales, couldn't replicate these effects.

Spectral Gaps and New Opportunities

The spectral gap emerges as a unique factor setting Facebook-100 apart from other low-informativeness datasets. This distinction is essential as it localizes effects to one-hop neighborhoods, consistently replicating across diverse architectures. It begs the question: are we underestimating the role of spectral properties in GNN performance?

existing benchmarks might not be numerically insufficient but are perhaps compositionally skewed. The Facebook-100 dataset offers a fresh target for testing and refining adaptive aggregation methods. It's a wake-up call for the industry, suggesting that reliance on conventional wisdom could be misleading.

Beyond Traditional Design Rules

One might wonder, why does this matter? Because it challenges the industry's prevailing assumption that GNN design generalizes across datasets. The findings underscore the importance of questioning whether established design rules truly hold up in diverse data environments. In some cases, a single-aggregator GIN can outperform more complex models like PNA on standard benchmarks.

The AI-AI Venn diagram is getting thicker. Facebook-100's insights are more than just anomalies. They're a call to action, urging us to reconsider how we perceive model adaptability and dataset composition. As we forge ahead, the question isn't whether these new insights will influence future GNN strategies, it's how quickly we can adapt.