The paper proposes a new graph neural network model called GRADE, which is based on aggregation-diffusion equations. The key insights are:
Aggregation-diffusion equations on graphs can exhibit metastable behavior, where node representations form multiple clusters and persist in these local equilibrium states for long periods before transitioning to the global equilibrium. This metastable effect helps alleviate the over-smoothing issue in traditional graph neural networks.
GRADE incorporates both nonlinear diffusion and interaction potentials, generalizing existing diffusion-based continuous graph neural networks. The flexible choices for diffusion coefficients and interaction kernels provide GRADE with strong expressive capabilities.
Theoretically, the authors prove that GRADE can mitigate over-smoothing, especially when using logarithmic interaction potentials. Existing diffusion-based models can be seen as linear degenerated versions of GRADE.
Experiments show that GRADE achieves competitive performance on various node classification benchmarks, including both homophilic and heterophilic datasets. Additionally, GRADE demonstrates an impressive ability to preserve Dirichlet energy, indicating its effectiveness in addressing the over-smoothing problem.
Til et andet sprog
fra kildeindhold
arxiv.org
Vigtigste indsigter udtrukket fra
by Kaiyuan Cui,... kl. arxiv.org 04-01-2024
https://arxiv.org/pdf/2403.20221.pdfDybere Forespørgsler