Dataflow-Aware PIM-Enabled Manycore Architecture for Efficient Acceleration of Deep Learning Workloads

Belangrijkste concepten

Designing dataflow-aware on-chip communication networks (NoI/NoC) is crucial for efficient acceleration of deep learning workloads on manycore architectures enabled by 2.5D/3D integration.

Samenvatting

The paper presents the design principles of a dataflow-aware manycore platform for accelerating various deep learning workloads using 2.5D and 3D integration technologies.
Key highlights:

2.5D chiplet-based systems offer a promising alternative to monolithic chips, but scalable communication between chiplets is challenging. The authors propose a dataflow-aware space-filling curve (SFC) based NoI architecture called Floret to address this.
Floret connects chiplets along a contiguous path to map neural layers efficiently, reducing latency and energy consumption compared to baseline NoI architectures like mesh, torus, and application-specific designs.
For 3D integration, the authors highlight the need to jointly optimize performance and thermal aspects to maintain DNN inference accuracy. A Floret-inspired 3D NoC architecture is proposed to address this.
The authors also discuss the unique challenges in designing dataflow-aware architectures for emerging ML workloads like Transformers, which exhibit heterogeneous computational kernels.

Statistieken

Deep neural networks like ResNet, VGG, and DenseNet can have 24.76 million to 93.4 million trainable parameters.
Floret outperforms baseline NoI architectures like Kite and SIAM by up to 2.24x in latency and 2.8x in energy consumption for a 100-chiplet 2.5D system.
Floret reduces the NoI fabrication cost by 2.8x, 2.1x, and 1.89x compared to Kite, SIAM, and SWAP, respectively, for a 100-chiplet system.
For a 100-PE 3D NoC system, the joint performance-thermal optimized design achieves 17°C lower peak temperature and 11% higher DNN inference accuracy compared to the performance-only optimized Floret-enabled NoC.

Citaten

"Simply allocating more resources (ReRAMs) to speed up only computation is ineffective if the communication infrastructure cannot keep up with it."
"Dataflow awareness during the design process is imperative so that the communicating neural layers are highly likely to run on neighboring chiplets/PEs without introducing a significant volume of long-range and multi-hop data exchange."
"As the complexity of deep neural networks (DNNs) grows, we must design manycore-based accelerators with multiple processing elements (PEs) on a single chip."

Belangrijkste Inzichten Gedestilleerd Uit

Dataflow-Aware PIM-Enabled Manycore Architecture for Deep Learning Workloads

by Harsh Sharma... om arxiv.org 03-29-2024

https://arxiv.org/pdf/2403.19073.pdf

Diepere vragen

How can the proposed dataflow-aware design principles be extended to handle dynamic workloads with varying DNN models and input sizes

The proposed dataflow-aware design principles can be extended to handle dynamic workloads with varying DNN models and input sizes by implementing a flexible mapping strategy. This strategy should prioritize contiguous mapping of neural layers to chiplets to reduce communication overhead. For dynamic workloads, a queue-based mapping approach can be adopted to assign DNN tasks sequentially, ensuring that each task is mapped to neighboring chiplets for efficient data exchange. Additionally, the design can incorporate adaptive routing algorithms that dynamically adjust the mapping based on the workload characteristics, such as varying input sizes and computational requirements. By considering the unique dataflow patterns of different DNN models and input sizes, the manycore architecture can optimize resource utilization and performance for dynamic workloads.

What are the potential challenges in integrating the heterogeneous computational kernels of emerging ML models like Transformers onto the dataflow-aware manycore architecture

Integrating the heterogeneous computational kernels of emerging ML models like Transformers onto the dataflow-aware manycore architecture poses several potential challenges. One key challenge is the diverse nature of computational requirements and memory access patterns in Transformer models, which may require specialized hardware modules such as Tensor cores, GPUs, and PIM-based accelerators. Ensuring seamless integration of these modules while maintaining dataflow awareness and minimizing communication overhead is crucial. Another challenge is the storage and processing of intermediate matrices in memory-intensive operations like multi-head self-attention, which may exceed the capacity of traditional PIM architectures. Addressing these challenges requires a holistic approach that combines dataflow-aware design principles with adaptive hardware configurations to support the computational diversity of Transformer models.

How can the thermal-aware design methodology be further improved to enable reliable operation of ReRAM-based PIM accelerators under diverse operating conditions

To further improve the thermal-aware design methodology for reliable operation of ReRAM-based PIM accelerators, several enhancements can be considered. Firstly, incorporating dynamic thermal management techniques that monitor and regulate the temperature of the ReRAM arrays in real-time can help mitigate thermal issues and maintain inference accuracy. This can involve adaptive power gating, workload scheduling, and temperature-aware resource allocation strategies. Additionally, optimizing the physical layout of the ReRAM-based PIM accelerators to enhance heat dissipation and reduce thermal gradients can improve overall system reliability. Furthermore, exploring advanced cooling solutions such as microfluidic cooling or on-chip heat sinks can effectively manage thermal hotspots and ensure consistent performance under diverse operating conditions. By integrating these advanced thermal management techniques, the design methodology can enhance the robustness and longevity of ReRAM-based PIM accelerators in datacenter-scale applications.