Approximation Properties of Differentiable Neural Networks with Rectified Power Unit Activation and Applications to Score Estimation and Isotonic Regression

Differentiable RePU neural networks can accurately represent multivariate polynomials and simultaneously approximate smooth functions and their derivatives, with improved approximation rates when the data has a low-dimensional structure. These properties enable applications in score estimation and isotonic regression.