Statistical Foundations of In-Context Learning for Linear Regression
Pretraining a linearly parameterized single-layer linear attention model for linear regression with a Gaussian prior requires only a small number of independent tasks.
Wie In-Context Learning Beziehungen zwischen Etiketten lernt, aber kein herkömmliches Lernen ist
In-Context Learning (ICL) kann tatsächlich Beziehungen zwischen Etiketten aus den Beispielen in Kontexten lernen, ist aber nicht mit konventionellem Lernen gleichzusetzen.
Transformers Mimic Bayesian Inference for In-Context Learning Across Linear and Nonlinear Function Classes
High-capacity transformers can mimic Bayesian inference when performing in-context learning across a diverse range of linear and nonlinear function classes. The inductive bias of in-context learning is determined by the pretraining data distribution.
Enhancing Natural Language Processing with In-Context Learning: A Comprehensive Survey
In-context learning (ICL) has emerged as a powerful paradigm for natural language processing, enabling large language models to make predictions based on a few demonstration examples. This survey aims to comprehensively review the progress and challenges of ICL.
대규모 언어 모델은 지식 추론을 위한 맥락 내 교사가 될 수 있는가: 인코딩 특이성 가설과 셀프 설명, 티치백 프롬프팅 소개
대규모 언어 모델(LLM)은 인간이 작성한 것보다 더 효과적인 맥락 내 추론 예시를 생성하여 더 효과적으로 지식 추론 능력을 향상시킬 수 있다.
Density Estimation with Large Language Models: Analyzing In-Context Learning Trajectories Using Intensive Principal Component Analysis
Large language models (LLMs) exhibit an inherent ability for density estimation, effectively approximating probability density functions from in-context data through a mechanism resembling adaptive kernel density estimation.
LLM을 이용한 밀도 추정: In-Context 학습 궤적에 대한 기하학적 연구
대규모 언어 모델(LLM)은 In-Context 학습을 통해 확률 밀도 함수(PDF)를 추정하는 능력을 보이며, 이는 적응형 커널 밀도 추정(KDE)으로 해석될 수 있습니다.
Understanding In-Context Learning in Large Language Models: A Two-Dimensional Coordinate System Based on Task Recognition and Example Similarity
In-context learning (ICL) effectiveness in large language models (LLMs) depends on the interplay between the model's ability to recognize the task and the presence of similar examples in demonstrations, forming four distinct scenarios within a two-dimensional coordinate system.
The Effect of Examples on In-Context Learning: A Theoretical Case Study for Binary Classification in Large Language Models
The accuracy of in-context learning (ICL) in large language models (LLMs) for binary classification tasks is a complex interplay between pre-training knowledge, the number and quality of in-context examples, and potential dependencies among examples.
In-Context Learning in Neural Networks: Exploring Curriculum Effects and Compositionality
Neural networks capable of both in-context learning (ICL) and traditional in-weight learning (IWL) can exhibit dual learning behaviors observed in humans: demonstrating compositional generalization and a blocking advantage in rule-governed tasks, while exhibiting an interleaving advantage in tasks lacking such structure.
