Core Concepts
Designing biased over-the-air federated learning solutions can provide higher gains over existing zero-bias schemes in heterogeneous wireless settings by optimizing the trade-off between model bias and variance.
Abstract
The paper studies the performance of over-the-air federated learning (OTA-FL) in a wireless network with heterogeneous device conditions. Unlike prior works that assume homogeneous wireless settings, the authors consider a more practical scenario where devices experience different average path losses.
Key highlights:
- The authors derive an upper bound on the model "optimality error" that explicitly captures the effect of bias and variance in the OTA-FL updates.
- Based on this bound, they propose two OTA device pre-scaler designs: 1) minimum noise variance, and 2) minimum noise variance zero-bias solutions.
- Numerical results show that the proposed minimum noise variance biased pre-scalers can provide significant gains in global loss and test accuracy compared to existing schemes under wireless heterogeneity.
- The analysis reveals that forcing zero-bias in each round, as done in prior works, may lead to high variance in the FL updates under heterogeneous wireless conditions, motivating the need for biased OTA-FL designs.
Stats
The average of the squared norm of the local gradients at the global minimizer w* is bounded by κ^2.
The norm of local gradients in each FL round is uniformly bounded by Gmax.
Quotes
"To optimize this trade-off, we study the design of OTA device pre-scalers by focusing on the OTA-FL convergence."
"Numerical evaluations show that using OTA device pre-scalers that minimize the variance of FL updates, while allowing a small bias, can provide high gains over existing schemes."