
A simple combination of Low-Rank Adaptation (LoRA) and Gaussian Stochastic Weight Averaging (SWAG) can effectively enable approximate Bayesian inference in large language models, improving their generalization and calibration.


coremsg

efficient-bayesian-adaptation-of-large-language-models-using-gaussian-stochastic-weight-averaging-and-low-rank-adaptation


Efficient Bayesian Adaptation of Large Language Models Using Gaussian Stochastic Weight Averaging and Low-Rank Adaptation


title_rewrite


Bayesian Low-Rank Adaptation by Backpropagation (BLoB) jointly estimates the mean and covariance of the variational distribution of large language model parameters during fine-tuning, improving generalization and uncertainty estimation.


bayesian-low-rank-adaptation-by-backpropagation-for-improving-uncertainty-estimation-in-large-language-models


Bayesian Low-Rank Adaptation by Backpropagation for Improving Uncertainty Estimation in Large Language Models
