Limitations of Training Data Mixture in Ensuring Out-of-Distribution Generalization of Deep Neural Networks
Simply increasing the size of training data mixture cannot guarantee the out-of-distribution generalization ability of deep neural networks. The generalization error can exhibit diverse non-decreasing trends depending on the degree of distribution shift between training and test data.