Generalization Bounds for Deep Neural Networks Based on Information Theory

The core message of this work is to derive new hierarchical generalization error bounds for deep neural networks (DNNs) using information-theoretic measures. The bounds capture the effect of network depth and quantify the contraction of relevant information measures as the layer index increases, highlighting the benefits of deep models for learning.