A Geometric Explanation of the Paradox in Likelihood-Based Out-of-Distribution Detection with Deep Generative Models

Likelihood-based deep generative models can assign higher likelihoods to out-of-distribution (OOD) data from simpler sources, despite never generating such OOD samples. This paradox can be explained by the fact that these models assign high density but low probability mass to regions containing the OOD data, which have lower intrinsic dimension than the in-distribution data.