Architectural Biases in Untrained Neural Networks: How Model Design Choices Can Skew Initial Predictions

Architectural choices in neural network design, such as activation functions, pooling layers, and data preprocessing, can introduce an inherent bias in the initial predictions of untrained models, even in the absence of explicit biases in the data or training process.