Maintaining Plasticity in Deep Continual Learning: A Systematic Study of Loss of Plasticity in Modern Deep Neural Networks
Deep learning methods suffer from a fundamental loss of plasticity in continual learning problems, severely limiting their ability to adapt to changing data distributions.
Enhancing Deep Neural Network Training with Noise-Based Learning Algorithms
Noise-based learning algorithms, such as node perturbation (NP), can provide an efficient alternative to backpropagation (BP) for training deep neural networks. By combining different NP formulations with a decorrelation mechanism, the performance of NP-based learning can be significantly improved, approaching or even exceeding that of BP in certain contexts.
Improving Computational Efficiency of Convolutional Neural Networks through Block Fusion
Computational efficiency, not just model efficiency, is crucial for achieving high-performance convolutional neural networks. By co-optimizing model efficiency and computational efficiency through block fusion, it is possible to create models that are both accurate and fast.
Generalization Bounds for Deep Neural Networks Based on Information Theory
The core message of this work is to derive new hierarchical generalization error bounds for deep neural networks (DNNs) using information-theoretic measures. The bounds capture the effect of network depth and quantify the contraction of relevant information measures as the layer index increases, highlighting the benefits of deep models for learning.
Learning in Convolutional Neural Networks Accelerated by Integrating Transfer Entropy
Integrating transfer entropy (TE) feedback connections into the training process of convolutional neural networks (CNNs) can accelerate the training process and improve the classification accuracy, at the cost of increased computational overhead.
GenQ: Quantization in Low Data Regimes with Generative Synthetic Data
GenQ introduces a novel approach using Generative AI models to generate synthetic data for quantization, setting new benchmarks in data-free and data-scarce quantization.
Lernen aus der Vergangenheit: Ein Proxy-geführtes Verteidigungsrahmenwerk mit Selbstdestillationsregulierung
Die Nutzung historischer Zustände des Zielmodells verbessert die Robustheit und Stabilität von Deep Learning-Modellen.
LumiNet: Enhancing Logit-Based Knowledge Distillation with Perception
LumiNet introduces a novel approach to knowledge distillation by enhancing logit-based distillation with the concept of 'perception', addressing overconfidence issues and improving knowledge extraction.
Universality of Linear Recurrences Followed by Non-linear Projections: Insights and Validation
Kombination von linearen RNNs und MLPs ermöglicht universelle Approximation von Sequenz-zu-Sequenz-Abbildungen.
DeepTextMark: A Deep Learning-Driven Text Watermarking Approach for Identifying Large Language Model Generated Text
DeepTextMark introduces a deep learning-driven text watermarking approach for identifying text generated by Large Language Models, emphasizing blind, robust, reliable, automatic, and imperceptible characteristics.
