The paper discusses how principles from information retrieval (IR) research can be applied to improve the effectiveness of in-context learning (ICL), a new paradigm in natural language processing where a small number of examples are appended to a prompt to control the text generation process of a large language model.
The key ideas proposed are:
Adaptive ICL (AICL): Instead of using a fixed number of examples, the number of examples can be dynamically selected based on the predicted usefulness of the examples. This can be done using unsupervised approaches inspired by query performance prediction techniques in IR, or a supervised approach that learns to predict the optimal number of examples.
Supervised Ranking for Example Selection: The notion of relevance in IR can be adapted to define the "usefulness" of examples for the downstream ICL task. Supervised ranking models, such as bi-encoders and cross-encoders, can be trained to rank the examples based on their predicted usefulness.
Diversifying Examples: Inspired by faceted search and diversified ranking in IR, the paper suggests that providing diverse examples to the ICL model can help prevent biases and improve the coverage of different aspects relevant to the downstream task.
The paper also includes a preliminary evaluation showing the benefits of adaptive ICL compared to using a fixed number of examples.
To Another Language
from source content
arxiv.org
Viktige innsikter hentet fra
by Andrew Parry... klokken arxiv.org 05-03-2024
https://arxiv.org/pdf/2405.01116.pdfDypere Spørsmål