Core Concepts
A new normal mapbased proximal random reshuffling (normPRR) method is proposed for solving nonsmooth nonconvex finitesum optimization problems. NormPRR achieves improved iteration complexity bounds compared to existing proximaltype random reshuffling methods, and also exhibits strong asymptotic convergence guarantees.
Abstract
The paper presents a new proximal random reshuffling algorithm called normPRR for solving nonsmooth nonconvex finitesum optimization problems. The key contributions are:

Complexity Analysis:
 NormPRR achieves an iteration complexity of O(n^(1/3)T^(2/3)) in expectation, improving over the currently known bounds for this class of problems.
 NormPRR also has a deterministic complexity bound of O(T^(2/3)).
 These complexity results match the best known bounds for random reshuffling methods in the smooth nonconvex setting.

Asymptotic Convergence:
 Under suitable step size conditions, normPRR is shown to converge globally, with the stationarity measure dist(0, ∂ψ(wk)) converging to 0 and the objective function ψ(wk) converging to an optimal value.
 For diminishing step sizes, the whole sequence of iterates {wk} is proven to converge to a single stationary point.
 Quantitative asymptotic convergence rates are derived that can match those in the smooth, strongly convex setting.

Numerical Experiments:
 Experiments on nonconvex classification tasks demonstrate the efficiency of the proposed normPRR approach.
The key innovation of normPRR is the use of the normal map, which allows for better compatibility with withoutreplacement sampling schemes compared to existing proximaltype random reshuffling methods.
Stats
∥∇f(w, i)∥^2 ≤ 2L[f(w, i)  f_lb]
σ^2_k ≤ 2L[ψ(wk)  ψ_lb]
Quotes
"Random reshuffling techniques are prevalent in largescale applications, such as training neural networks. While the convergence and acceleration effects of random reshufflingtype methods are fairly well understood in the smooth setting, much less studies seem available in the nonsmooth case."
"We show that normPRR achieves the iteration complexity O(n^(1/3)T^(2/3)) where n denotes the number of component functions f(·, i) and T counts the total number of iterations. This improves the currently known complexity bounds for this class of problems by a factor of n^(1/3)."
"Moreover, we derive last iterate convergence rates that can match those in the smooth, strongly convex setting."