Rethinking Relational Learning: The Geometry Factor

Relational learning has long leaned on flat leaderboards that average results across various datasets, assuming a uniform structure. But the reality is different. This approach introduces bias, obscuring how geometry affects performance and leading to potentially flawed conclusions about generalization.

Unveiling Geometry's Role

The study identifies intrinsic geometry as a important factor in model effectiveness. Conventional metrics hide performance trade-offs that only emerge when datasets are grouped by geometric properties. This revelation has significant implications for how we evaluate models.

To address this, researchers proposed a curvature-stratified evaluation framework, dividing datasets into positive, negative, and near-zero curvature categories. This method reveals that model rankings remain stable within each category but vary significantly across them.

The Battle of the Models

The benchmark tested 18 models, including Graph Convolutional Networks (GCNs), Graph Foundation Models (GFMs), and tabular learning methods, across 14 datasets. The findings suggest that geometry, rather than a universal solution, dictates performance.

Interestingly, GFMs sometimes offer diminishing returns compared to geometry-aligned GNNs. Strip away the marketing, and you get a clear message: performance isn't one-size-fits-all.

The Future of Evaluations

What does this mean for future evaluations? A geometry-aware protocol could provide more reliable and interpretable comparisons than traditional aggregated benchmarks. Why settle for less when a more nuanced approach is available?

The team has made all code, dataset splits, and evaluation tools publicly available, encouraging reproducible and rigorous assessment of relational learning methods. The numbers tell a different story when geometry is considered.