Characterizing the Function Space Explored by Multi-Layer Neural Networks

The author proposes a function space called the Neural Hilbert Ladder (NHL) that can characterize the functions representable by multi-layer neural networks with arbitrary width. The NHL space is defined as an infinite union of reproducing kernel Hilbert spaces (RKHSs) and is associated with a complexity measure that governs both the approximation and generalization properties of neural networks.