Основные понятия
Bayesian neural networks can achieve better generalization performance by explicitly seeking flat posteriors during optimization, leading to more effective Bayesian Model Averaging.
Lim, S., Yeom, J., Kim, S., Byun, H., Kang, J., Jung, Y., ... & Song, K. (2024). Flat Posterior Does Matter for Bayesian Model Averaging. ICLR 2025.
This paper investigates the impact of loss landscape flatness on the performance of Bayesian neural networks (BNNs), particularly in the context of Bayesian Model Averaging (BMA). The authors aim to demonstrate that BNNs often struggle to capture flat minima and that this limitation hinders the effectiveness of BMA. To address this, they propose a novel optimization algorithm called Sharpness-Aware Bayesian Model Averaging (SA-BMA) designed to explicitly seek flat posteriors.