The paper introduces neural graph features (GRAF) as a simple-to-compute set of properties of architectural graphs that can be used for efficient performance prediction in neural architecture search (NAS).
The authors first examine the limitations of existing zero-cost proxies, showing that many of them directly depend on the number of convolutions in the network rather than capturing more complex structural properties. Inspired by this, the authors propose GRAF, which includes features like operation counts, path lengths, and node degrees.
When used as input to a random forest predictor, GRAF outperforms zero-cost proxies and other common encodings like one-hot representations, especially on smaller training sets. The combination of GRAF and zero-cost proxies achieves the best overall performance, outperforming most existing predictors at a fraction of the computational cost.
The interpretability of GRAF also allows the authors to analyze which network properties are important for different tasks. For example, skip connections and convolution path lengths are crucial for image classification tasks, while node degree features are more important for other domains like autoencoding.
The authors further evaluate GRAF on a variety of tasks beyond just validation accuracy prediction, including hardware metrics and robustness. GRAF demonstrates strong performance across these diverse settings as well. Finally, they show that GRAF can also improve the performance of more complex predictors like BRP-NAS when used as additional input features.
Til et annet språk
fra kildeinnhold
arxiv.org
Viktige innsikter hentet fra
by Gabr... klokken arxiv.org 04-26-2024
https://arxiv.org/pdf/2404.16551.pdfDypere Spørsmål