
Pre-trained language models' base capabilities are influenced by architecture, with FFN-Wider Transformers reducing the contribution ratio of the combination function, leading to a decline in base capabilities.


coremsg

ffn-wider-transformer-models-and-architecture-influence-on-language-models


FFN-Wider Transformer Models and Architecture Influence on Language Models


title_rewrite


Architecture, specifically the FFN-Wider Transformer model, significantly impacts the base capabilities of language models by altering the contribution ratio of combination and transformation functions.


ffn-wider-transformer-architecture-impact-on-language-models


FFN-Wider Transformer Architecture Impact on Language Models
