Core Concepts
The author proposes an automated augmentation flow to convert existing models into Early Exit Neural Networks (EENNs) for improved efficiency in heterogeneous or distributed hardware targets.
Abstract
The content discusses the development of a framework that automates the conversion of standard models into EENNs, enhancing efficiency in IoT environments. The framework addresses challenges in designing EENNs and aims to make them accessible to developers without specialized knowledge. It evaluates the approach on various use cases, showcasing significant reductions in mean operations per inference and energy consumption.
Stats
For a speech command detection task, the solution reduced mean operations per inference by 59.67%.
For an ECG classification task, it terminated all samples early, reducing mean inference energy by 74.9% and computations by 78.3%.
On CIFAR-10, the solution achieved up to a 58.75% reduction in computations.
The search on a ResNet-152 base model for CIFAR-10 took less than nine hours on a laptop CPU.
Quotes
"The proposed framework constructs the EENN architecture, maps subgraphs to hardware targets, and configures decision mechanisms automatically."
"Our solution showcased significant reductions in mean operations per inference and energy consumption across various use cases."