Computational Limits and Efficient Variants of Modern Hopfield Models
The computational limits of modern Hopfield models are characterized by a norm-based phase transition, where efficient sub-quadratic variants exist only when the norms of input query and memory patterns are below a certain threshold. An efficient nearly linear-time modern Hopfield model is provided as an example, maintaining exponential memory capacity.
