Theoretical Expressive Power and Design Space of Higher-Order Graph Transformers

This paper provides a systematic study of the theoretical expressive power of order-k graph transformers and their sparse variants. It shows that a plain order-k graph transformer without additional structural information is less expressive than the k-Weisfeiler Lehman (k-WL) test, but adding explicit tuple indices can make it as expressive as k-WL. The paper then explores strategies to sparsify and enhance the higher-order graph transformers, aiming to improve both their efficiency and expressiveness.