Efficient Fine-Tuning with Model Stock: Leveraging Geometric Properties of Weight Space for Improved Performance

By leveraging the geometric properties of fine-tuned weights, Model Stock approximates the center of the weight distribution using only a few fine-tuned models, achieving superior in-distribution and out-of-distribution performance compared to existing methods.