
Differentiable pruning combined with combinatorial optimization enhances structured neural network pruning.


coremsg

sequentialattention-for-block-sparsification-combining-differentiable-pruning-and-combinatorial-optimization


SequentialAttention++ for Block Sparsification: Combining Differentiable Pruning and Combinatorial Optimization


title_rewrite


SPA is a versatile structured pruning framework that can prune neural networks with any architecture, from any framework, and at any stage of training, achieving state-of-the-art pruning results without the need for fine-tuning or calibration data.


structurally-prune-any-neural-network-across-frameworks-and-architectures


Structurally Prune Any Neural Network Across Frameworks and Architectures



The authors propose a novel method for structured pruning of pre-trained deep neural networks that involves projecting unit activations to an orthogonal subspace, ranking units based on their non-redundant variance, and using a global variance-based cutoff to automatically determine layer-wise pruning ratios.


efficient-neural-network-pruning-via-subspace-factorization-and-variance-based-importance-scoring


Efficient Neural Network Pruning via Subspace Factorization and Variance-Based Importance Scoring
