
Neural network landscapes exhibit surprising connectivity and geometric properties, including star-shaped connectivity between global minima and near-convex geodesic distances, even for highly non-convex models.


coremsg

exploring-the-connectivity-and-geometry-of-neural-network-landscapes-star-shaped-paths-and-geodesic-distances


Exploring the Connectivity and Geometry of Neural Network Landscapes: Star-Shaped Paths and Geodesic Distances


title_rewrite


The authors propose a structure-guided Gauss-Newton (SgGN) method that effectively utilizes both the least squares structure and the neural network structure of the objective function to solve optimization problems involving shallow ReLU neural networks.


efficient-structure-guided-gauss-newton-method-for-shallow-relu-neural-network-optimization


Efficient Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network Optimization



A genetic algorithm-based quantization-aware approximation method (GQA-LUT) is proposed to efficiently handle diverse non-linear operations in Transformers using integer-only arithmetic.


genetic-quantization-aware-approximation-for-efficient-integer-only-inference-of-non-linear-operations-in-transformers


Genetic Quantization-Aware Approximation for Efficient Integer-Only Inference of Non-Linear Operations in Transformers
