核心概念
Hierarchical Skip Decoding(HSD)は、効率的な自己回帰テキスト生成を実現する新しいデコーディング戦略です。
統計
GPT-2 + CALM d = 0.02:R-L 0.71, R-L 6.38, BERTScore 9.40
Phi-2 + CALM d = 0.005:R-L 0.93, R-L 11.87, BERTScore 10.71
引用
"Hierarchical Skip Decoding (HSD) integrates the concepts of descending scheduled forward layers and hierarchical layer skipping."
"HSD skips the decoding layer in a scheduled and hierarchical manner."
"HSD strives for enhancing the overall text generation quality while utilizing a similar level of computational resources."