The paper discusses how principles from information retrieval (IR) research can be applied to improve the effectiveness of in-context learning (ICL), a new paradigm in natural language processing where a small number of examples are appended to a prompt to control the text generation process of a large language model.
The key ideas proposed are:
Adaptive ICL (AICL): Instead of using a fixed number of examples, the number of examples can be dynamically selected based on the predicted usefulness of the examples. This can be done using unsupervised approaches inspired by query performance prediction techniques in IR, or a supervised approach that learns to predict the optimal number of examples.
Supervised Ranking for Example Selection: The notion of relevance in IR can be adapted to define the "usefulness" of examples for the downstream ICL task. Supervised ranking models, such as bi-encoders and cross-encoders, can be trained to rank the examples based on their predicted usefulness.
Diversifying Examples: Inspired by faceted search and diversified ranking in IR, the paper suggests that providing diverse examples to the ICL model can help prevent biases and improve the coverage of different aspects relevant to the downstream task.
The paper also includes a preliminary evaluation showing the benefits of adaptive ICL compared to using a fixed number of examples.
In un'altra lingua
dal contenuto originale
arxiv.org
Approfondimenti chiave tratti da
by Andrew Parry... alle arxiv.org 05-03-2024
https://arxiv.org/pdf/2405.01116.pdfDomande più approfondite