Advocating for Retrieval-Augmented Language Models Over Parametric LMs
Retrieval-augmented language models offer reliability, adaptability, and attributability over parametric models. The paper advocates for their widespread adoption through advancements in architecture, training methodologies, and infrastructure.
Investigating the Impact of Retrieval Augmentation on Language Model Capabilities
Retrieval-augmented language models separate linguistic knowledge from world knowledge, with larger models exhibiting a more pronounced separation. However, this improvement in syntactic understanding comes at the cost of reduced performance in general language understanding tasks that require resolving long-range context dependencies.
Enhancing Language Models with Retrieval-Augmented Generation (RAG) 2.0
Retrieval-Augmented Generation (RAG) is a technique to enhance language models by providing additional context, enabling them to generate more specific and informative responses.
Efficient Retrieval-Augmented Language Models with Binary Token Representations
Binary token representations can significantly improve the inference speed and reduce the storage footprint of retrieval-augmented language models while maintaining high task performance.
Improving Retrieval-Augmented Language Models' Robustness to Irrelevant Context
Retrieval-augmented language models can be made more robust to irrelevant retrieved context through a combination of natural language inference-based filtering and fine-tuning on a mixture of relevant and irrelevant contexts.
Enhancing the Robustness of Retrieval-Augmented Language Models with CHAIN-OF-NOTE (CON)
CHAIN-OF-NOTE (CON), a novel framework for Retrieval-Augmented Language Models (RALMs), enhances their robustness by generating sequential reading notes for retrieved documents, enabling better assessment of relevance and integration of external knowledge for more accurate and reliable responses.
Parenting: A Novel Framework for Enhancing Knowledge Selection in Retrieval-Augmented Language Models by Decoupling Adherence and Robustness Parameters
Parenting, a novel framework, enhances the knowledge selection process in Retrieval-Augmented Language Models (RALMs) by decoupling parameters related to adherence and robustness, leading to a more balanced and effective integration of external knowledge.
© 2024 by Linnk AI