Enhancing In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
Retrieval-augmented encoder-decoder language models can significantly improve in-context learning performance through a combination of retrieval-augmented masked language modeling, retrieval-augmented prefix language modeling, and Fusion-in-Context Learning.