Deep learning methods suffer from a fundamental loss of plasticity in continual learning problems, severely limiting their ability to adapt to changing data distributions.
Noise-based learning algorithms, such as node perturbation (NP), can provide an efficient alternative to backpropagation (BP) for training deep neural networks. By combining different NP formulations with a decorrelation mechanism, the performance of NP-based learning can be significantly improved, approaching or even exceeding that of BP in certain contexts.
Computational efficiency, not just model efficiency, is crucial for achieving high-performance convolutional neural networks. By co-optimizing model efficiency and computational efficiency through block fusion, it is possible to create models that are both accurate and fast.
The core message of this work is to derive new hierarchical generalization error bounds for deep neural networks (DNNs) using information-theoretic measures. The bounds capture the effect of network depth and quantify the contraction of relevant information measures as the layer index increases, highlighting the benefits of deep models for learning.
Integrating transfer entropy (TE) feedback connections into the training process of convolutional neural networks (CNNs) can accelerate the training process and improve the classification accuracy, at the cost of increased computational overhead.
GenQ introduces a novel approach using Generative AI models to generate synthetic data for quantization, setting new benchmarks in data-free and data-scarce quantization.
Die Nutzung historischer Zustände des Zielmodells verbessert die Robustheit und Stabilität von Deep Learning-Modellen.
LumiNet introduces a novel approach to knowledge distillation by enhancing logit-based distillation with the concept of 'perception', addressing overconfidence issues and improving knowledge extraction.
Kombination von linearen RNNs und MLPs ermöglicht universelle Approximation von Sequenz-zu-Sequenz-Abbildungen.
DeepTextMark introduces a deep learning-driven text watermarking approach for identifying text generated by Large Language Models, emphasizing blind, robust, reliable, automatic, and imperceptible characteristics.