Główne pojęcia
This paper introduces an accelerated optimization framework for solving the linear-quadratic regulator (LQR) problem, which is a landmark problem in optimal control. The authors present novel continuous-time and discrete-time algorithms that achieve Nesterov-optimal convergence rates for the state-feedback LQR (SLQR) problem. For the output-feedback LQR (OLQR) problem, a Hessian-free accelerated framework is proposed that can find an ϵ-stationary point with second-order guarantee in a time of O(ϵ^(-7/4) log(1/ϵ)).
Streszczenie
The paper introduces an accelerated optimization framework for solving the linear-quadratic regulator (LQR) problem, which is a fundamental problem in optimal control. The authors distinguish between the state-feedback LQR (SLQR) and output-feedback LQR (OLQR) problems based on whether the full state is available.
For the SLQR problem:
- The authors prove the Lipschitz Hessian property of the LQR performance criterion, which is crucial for the application of modern optimization techniques.
- A continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order.
- A Nesterov-type method with a restarting rule is proposed for the discrete-time algorithm, which preserves the continuous-time convergence rate.
For the OLQR problem:
- A Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation.
- The method can find an ϵ-stationary point of the performance criterion in a time of O(ϵ^(-7/4) log(1/ϵ)), which improves upon the O(ϵ^(-2)) complexity of vanilla gradient descent.
- The method provides the second-order guarantee of stationary point.
The key contributions of the paper include:
- Proving the Lipschitz Hessian property of the LQR performance criterion, which is essential for the convergence analysis.
- Proposing an accelerated optimization framework for the SLQR problem, with both continuous-time and discrete-time algorithms achieving Nesterov-optimal convergence rates.
- Developing a Hessian-free accelerated framework for the OLQR problem, which can find an ϵ-stationary point with second-order guarantee in an improved time complexity.