
This research paper argues that relying on manually selected label token probabilities for classification in In-Context Learning (ICL) is suboptimal and proposes a novel method called Hidden Calibration, which utilizes centroid classifiers on the last hidden states of language models, leading to significant performance improvements in ICL.


coremsg

hidden-calibration-a-novel-approach-to-in-context-learning-outperforms-token-based-methods


Hidden Calibration: A Novel Approach to In-Context Learning Outperforms Token-based Methods


title_rewrite


In-context learning (ICL) can lead to miscalibration, especially in low-shot settings, and methods aimed at improving usability like fine-tuning and chain-of-thought prompting can further degrade calibration.


evaluating-the-calibration-of-in-context-learning-in-language-models


Evaluating the Calibration of In-context Learning in Language Models
