Analysis of Loopless Local Gradient Descent with Varying Step Size for Federated Learning
The article proposes L2GDV, a novel federated learning algorithm utilizing a varying step size in stochastic gradient descent, to efficiently optimize regularized empirical risk minimization problems while reducing communication costs.
Local Superior Soups: An Efficient Federated Learning Method for Pre-trained Models Under Data Heterogeneity
Local Superior Soups (LSS) is a novel method that leverages model interpolation and regularization techniques to enhance the efficiency of pre-trained models in federated learning, particularly in scenarios with data heterogeneity and limited communication rounds.
Identifying and Mitigating LoRA's Bottlenecks in Federated Fine-Tuning of Large Language Models
Direct weight aggregation, particularly when combined with the GaLore optimizer for local training, outperforms LoRA-based methods in federated fine-tuning of large language and vision transformer models, offering superior performance, stability, and generalization capabilities.
ProgFed: A Federated Learning Framework for Efficient Training by Network Growing
ProgFed is a novel federated learning framework that reduces communication and computation costs by progressively training increasingly complex models, achieving comparable or superior performance to traditional methods.
Efficient Adaptive Federated Optimization with Zero Local Preconditioner Initialization and Memory-Efficient Client Optimizers
FedAda2, a novel approach to federated learning, achieves efficient joint server- and client-side adaptive optimization by initializing local preconditioners from zero and employing memory-efficient client optimizers, thereby mitigating communication bottlenecks and client resource constraints without sacrificing performance.
Mixed-Precision Over-The-Air Federated Learning for Enhanced Performance and Energy Efficiency
Leveraging mixed-precision quantization and over-the-air aggregation in federated learning significantly improves both performance and energy efficiency, particularly in resource-constrained edge computing environments.
Why Go Full? Addressing Layer Mismatch in Federated Learning with Partial Network Updates
Partial network updates in federated learning, as opposed to full network updates, can mitigate layer mismatch issues, leading to faster convergence, improved accuracy, and reduced communication and computational overhead.
Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning Over Mobile Edge Networks
This paper proposes a novel online control scheme for client scheduling and resource allocation in Federated Learning (FL) over mobile edge networks, aiming to minimize training latency and enhance model accuracy under resource constraints and uncertainty.
A Practical Evaluation and Optimization of the FedNL Algorithm for Federated Learning
This paper presents a compute-optimized implementation of the FedNL algorithm family for federated learning, demonstrating significant speedups over the original implementation and existing solutions for logistic regression.
Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation (OTA Fed-Sophia)
The article introduces OTA Fed-Sophia, a novel second-order federated learning algorithm that leverages sparse Hessian estimation and over-the-air aggregation to achieve faster convergence with reduced communication costs and enhanced privacy for large-scale models.
© 2024 by Linnk AI