Tiny Models as Computational Savers for Large Models: TinySaver Approach
Tiny models can efficiently reduce computational demands for large models through the innovative TinySaver approach.
T3DNet: Compressing Point Cloud Models for Lightweight 3D Recognition
T3DNet proposes a two-stage method for compressing 3D point cloud models, achieving high compression rates without significant accuracy sacrifice.
Efficient Lossless and Tunable Lossy Compression Techniques for Large Foundation Models
Lossless and tunable lossy compression techniques can significantly reduce the storage and network bandwidth requirements of large foundation models without compromising their accuracy.
Convex Distillation: Compressing Deep Neural Networks Using Convex Optimization for Efficient Edge Deployment
This paper introduces a novel model compression technique called Convex Distillation, which leverages convex optimization to compress large, non-convex deep neural networks into smaller, more efficient convex networks, achieving comparable performance while eliminating the need for post-compression fine-tuning on labeled data.
DeltaDQ: Achieving Ultra-High Compression Ratios for Fine-Tuned Large Language Models Using Group-wise Dropout and Separate Quantization
DeltaDQ is a novel compression framework that significantly reduces the memory footprint of fine-tuned large language models (LLMs) while maintaining accuracy, enabling the deployment of multiple models on resource-constrained hardware.
Self-Data Distillation for Recovering Quality in Pruned Large Language Models: A Study on Llama3.1-8B Instruct
Self-data distillation, a novel fine-tuning technique leveraging the original unpruned model to generate a distilled dataset, effectively mitigates quality degradation in pruned large language models, outperforming standard supervised fine-tuning methods.
Similarity Guided Fast Layer Partition Pruning (SGLP) for Compressing Large Deep Learning Models
SGLP is a novel layer pruning method that leverages representation similarity and efficient importance evaluation to compress large deep learning models, achieving a balance between model size, computational efficiency, and performance.
Accurate Post-Training Pruning of Foundation Models via Multiple Removal Problem
This research paper introduces a novel post-training pruning method for foundation models that outperforms existing techniques by directly formulating and solving the Multiple Removal Problem (MRP), enabling simultaneous pruning of multiple weights and achieving higher accuracy without retraining.
© 2024 by Linnk AI