On the Turing Completeness and Efficiency of Prompting in Finite-Sized Transformers

This research paper theoretically proves that prompting a single, fixed-size Transformer can be Turing complete, meaning it can theoretically compute any computable function, achieving near-optimal computational efficiency comparable to the entire class of unbounded-size Transformers.