Conceitos essenciais
Reusing deep learning models can reduce engineering costs, but faces significant technical and decision-making challenges that require new software engineering practices and tools.
Resumo
This paper discusses the challenges of reusing deep neural networks (DNNs) in software engineering. DNNs have demonstrated exceptional performance in various domains, but their development and computational costs remain significant. Reusing DNNs can reduce these costs, but it presents its own set of challenges.
The authors identify three main types of DNN reuse:
Conceptual reuse: Replicating and re-engineering DNN algorithms, architectures, and concepts from academic literature and research prototypes.
Adaptation reuse: Leveraging existing pre-trained DNN models and adapting them to solve different learning tasks, e.g., through transfer learning or knowledge distillation.
Deployment reuse: Converting and deploying pre-trained DNN models in different computational environments and frameworks.
For each type of reuse, the paper outlines the key challenges:
Conceptual reuse:
Achieving reproducibility of reported DNN results
Replicating and re-engineering DNN models, including challenges with model operationalization, portability of DL operations, and performance debugging
Adaptation reuse:
Technical challenges with adaptation techniques like accuracy, latency, fairness, and robustness
Decision-making challenges in model selection, performance discrepancies, and security/privacy risks
Deployment reuse:
Interoperability issues between DNN frameworks and hardware
Establishing trust in the DNN supply chain, including attacks that threaten the integrity of reused models
The paper concludes by proposing several research directions to address these challenges, including improved artifact evaluation, testing tools, model auditing, recommendation systems, and enhanced supply chain security for DNNs.
Estatísticas
"Deep neural networks (DNNs) achieve state-of-the-art performance in many areas, including computer vision, system configuration, and question-answering."
"DNNs are expensive to develop, both in intellectual effort (e.g., devising new architectures) and computational costs (e.g., training)."
"Reusing DNNs is a promising direction to amortize costs within a company and across the computing industry."
Citações
"As with any new technology, however, there are many challenges in re-using DNNs. These challenges include both missing technical capabilities and missing engineering practices."
"The engineering community lacks long-term experience in appropriate engineering methods for deep learning."
"Prior work has characterized the challenges of developing DL software, yet relatively little is known about DL reuse and the specific challenges engineers face when trying to reuse these models."