Kernekoncepter
Optimizing amortized variational inference using Sequential Monte Carlo for inclusive KL minimization.
Resumé
The article introduces SMC-Wake as an alternative to RWS for fitting amortized variational approximations. It proposes three gradient estimators that are unbiased and consistent. SMC-Wake fits variational distributions accurately, avoiding mass concentration issues seen in RWS. The background discusses the challenges of minimizing forward KL divergence and the circular pathology in RWS. Experiments show SMC-Wake outperforming RWS in various scenarios, including two moons model, MNIST digit learning, Gaussian hierarchical model, and galaxy spectra emulator.
Statistik
K = 100 particles used in SMC-Wake.
M = 100 LT-SMC runs with K = 100 particles each.
SNR set to 1/σ with σ = 0.1 for galaxy spectra emulator.
100 walkers used for MCMC in galaxy spectra emulator.
Citater
"SMC-Wake avoids degeneracy by proposing from the prior."
"Experiments show SMC-Wake outperforming RWS in various scenarios."
"SMC-Wake provides accurate variational approximations."