Latte Transformer introduces a latent attention mechanism that scales linearly with sequence length, providing a drop-in replacement for standard attention.
