핵심 개념
Existing dataset distillation methods may have unexpected storage costs and training times, prompting the need for a more comprehensive evaluation metric.
초록
The content discusses the challenges and innovations in dataset distillation methods, focusing on Distributional Dataset Distillation (D3) and Federated Distillation. It highlights the importance of evaluating distillation methods based on storage costs, downstream training efficiency, and recovery accuracy. The study compares various state-of-the-art methods on ImageNet-1K and ResNet18, showcasing the effectiveness of D3 in achieving compact representations with improved performance.
Directory:
- Introduction
- Large Datasets and Dataset Distillation
- Distributional Dataset Distillation (D3)
- Federated Distillation
- Evaluation Metrics and Results
- Related Work
통계
최근의 데이터 집약화 방법은 예상치 못한 저장 비용과 훈련 시간을 초래할 수 있습니다.
D3는 효율적인 표현을 달성하기 위해 데이터를 압축하는 새로운 데이터 집약화 방법을 제안합니다.
D3는 ImageNet-1K 및 ResNet18에서 다양한 최첨단 방법과 비교하여 탁월한 성능을 보여줍니다.
인용구
"Dataset distillation methods have achieved remarkable success in producing much smaller datasets with limited loss of downstream model performance."
"We propose a novel distillation framework with smaller memory footprint that distills datasets into distributions."
"Our method outperforms existing work under various storage budgets, showcasing state-of-the-art performance."