Efficient Zeroth-Order Bilevel Optimization via Gaussian Smoothing

This paper proposes a fully zeroth-order stochastic approximation method for solving bilevel optimization problems, where neither the upper/lower objective values nor their unbiased gradient estimates are available. The authors use Gaussian smoothing to estimate the first- and second-order partial derivatives of the functions with two independent block of variables, and establish non-asymptotic convergence analysis and sample complexity bounds for the proposed algorithm.