核心概念
Stable and Expressive Positional Encodings (SPE) is a novel architecture for processing eigenvectors that achieves both stability and high expressive power for graph neural networks.
要約
The paper introduces Stable and Expressive Positional Encodings (SPE), a novel architecture for processing eigenvectors that aims to address two key challenges with existing Laplacian-based positional encodings:
Non-uniqueness: There are many different eigendecompositions of the same Laplacian, leading to basis ambiguity.
Instability: Small perturbations to the Laplacian can result in completely different eigenspaces, leading to unpredictable changes in positional encodings.
To tackle these issues, SPE performs a "soft partitioning" of eigensubspaces in an eigenvalue-dependent way, achieving both stability (from the soft partition) and high expressive power (from dependency on both eigenvalues and eigenvectors).
Specifically:
SPE is provably stable, with the stability determined by the gap between the d-th and (d+1)-th smallest eigenvalues.
SPE can universally approximate basis invariant functions and is at least as expressive as existing methods in distinguishing graphs. It can also effectively count graph substructures.
Experiments show that SPE significantly outperforms other positional encoding methods on molecular property prediction tasks and demonstrates improved robustness to domain shifts.
There is a trade-off between stability, generalization, and expressive power, with more stable models generalizing better but having lower training performance.
Overall, SPE provides a principled approach to designing stable and expressive positional encodings for graph neural networks.
統計
Small perturbations to the Laplacian matrix can result in completely different eigenspaces.
The stability of SPE is determined by the gap between the d-th and (d+1)-th smallest eigenvalues.
引用
"Small perturbations to the input Laplacian should only induce a limited change of final positional encodings."
"Eigenvectors have special structures that must be taken into consideration when designing architectures that process eigenvectors."