Khái niệm cốt lõi
Efficiently prune neural networks with FLOP and sparsity constraints using the FALCON framework.
Tóm tắt
The content introduces the FALCON framework for network pruning, considering both FLOP and sparsity constraints. It proposes an ILP-based approach, showcasing superior accuracy compared to existing methods within fixed budgets. The multi-stage FALCON++ method further enhances accuracy without retraining. Gradual pruning evaluations demonstrate improved accuracy relative to competitors.
Thống kê
"For instance, for ResNet50 with 20% of the total FLOPs retained, our approach improves the accuracy by 48% relative to state-of-the-art."
"Notably, without retraining, FALCON prunes a ResNet50 network to just 1.2 billion FLOPs (30% of total FLOPs) with 73% test accuracy."
"Our experiments reveal that, given a fixed FLOP budget, our pruned models exhibit significantly better accuracy compared to other pruning approaches."
Trích dẫn
"Using problem structure (e.g., the low-rank structure of approx. Hessian), we can address instances with millions of parameters."
"Our experiments demonstrate that FALCON achieves superior accuracy compared to other pruning approaches within a fixed FLOP budget."