Core Concepts
Wasserstein metrics enhance dataset distillation by capturing distribution differences efficiently.
Abstract
Abstract:
Dataset Distillation (DD) condenses large datasets into smaller synthetic equivalents while maintaining model performance.
Introduces Wasserstein distance for distribution matching in DD, achieving state-of-the-art results.
Introduction:
DD aims to encapsulate vast dataset information into a compact synthetic set.
Various innovative approaches proposed for DD include gradient matching, trajectory matching, and curvature matching.
Data Extraction:
"Our method achieves SOTA performance on various benchmarks."
"Experiments show that our method achieves SOTA performance on various benchmarks."
Quotations:
"Wasserstein distance has been known for measuring the differences between distributions."
Related Work:
Categorizes DD methods into Performance Matching, Parameter Matching, and Distribution Matching.
Preliminary:
Defines notations and formulates the optimization problem for Dataset Distillation.
Method:
Utilizes Wasserstein barycenter to learn synthetic datasets by aligning feature spaces of pretrained models.
Experiments:
Conducted experiments on high-resolution datasets like ImageNette and ImageNet-1K under different IPC settings.
Comparison with Other Methods:
Our method consistently showed SOTA performance across different datasets compared to baseline methods.
Further questions:
Stats
"Our method achieves SOTA performance on various benchmarks."
Quotes
"Wasserstein distance has been known for measuring the differences between distributions."