Enabling Efficient Hybrid Systolic Computation in Shared L1-Memory Manycore Clusters

Core Concepts

The proposed hybrid architecture enables efficient systolic execution on shared-memory, multi-core architectures without compromising their general-purpose capabilities, performance, and programmability.

Abstract

The paper presents a hybrid systolic-shared-memory architecture that combines the benefits of systolic arrays and shared-L1-memory manycore clusters. The key contributions are: A flexible architecture where small and energy-efficient RISC-V cores act as the systolic array's processing elements (PEs) and can form diverse, reconfigurable systolic topologies through queues mapped in the cluster's shared memory. Two low-overhead RISC-V ISA extensions, Xqueue and Queue-linked registers (QLRs), that accelerate queue management in hardware, enabling efficient systolic execution on the shared-memory cluster. Exploration of the hybrid systolic-shared-memory execution models enabled by the combination of systolic dataflow and global communication, analyzing the involved trade-offs. Hybrid implementations of key computational kernels like matrix multiplication, 2D convolution, and Fast Fourier transform are presented. A full implementation of the hybrid architecture on the open-source MemPool shared-L1-memory cluster, including hardware and software. Evaluation shows up to 73% utilization and 65% better energy efficiency compared to the shared-memory baseline.

Stats

For an area increase of just 6%, the hybrid architecture can double MemPool's compute unit utilization, reaching up to 73%. In a 22 nm FDX technology, the hybrid architecture runs at 600 MHz with no frequency degradation and achieves up to 208 GOPS/W. Up to 63% of the power is spent in the PEs for the hybrid architecture.

Quotes

"Combining systolic communication capabilities and shared-memory flexibility unveils a software design space of unprecedented trade-offs." "In our hybrid architecture, the software stack can benefit from both regular systolic dataflow and global, concurrent communication."

Key Insights Distilled From

Enabling Efficient Hybrid Systolic Computation in Shared L1-Memory Manycore Clusters

by Sergio Mazzo... at arxiv.org 04-25-2024

https://arxiv.org/pdf/2402.12986.pdf

Deeper Inquiries

How can the hybrid architecture be extended to support dynamic reconfiguration of the systolic topology at runtime?

To support dynamic reconfiguration of the systolic topology at runtime, the hybrid architecture can incorporate additional hardware components and software mechanisms. One approach is to introduce a reconfigurable interconnect network that allows for flexible connections between processing elements (PEs). This network can be controlled by a software runtime system that dynamically adjusts the connections based on the workload requirements. By implementing programmable routing and switching elements in the interconnect, the topology can be reconfigured on-the-fly to adapt to different dataflow patterns. Additionally, the software runtime can manage the allocation and deallocation of memory-mapped queues to facilitate the communication between PEs in the systolic array. This dynamic reconfiguration capability enhances the flexibility and adaptability of the hybrid architecture, enabling it to efficiently handle a variety of workloads with changing computational requirements.

What are the potential limitations or drawbacks of the hybrid approach compared to pure systolic or pure shared-memory architectures?

While the hybrid approach offers a balance between the benefits of systolic arrays and shared-memory manycore clusters, it also comes with certain limitations and drawbacks. One potential limitation is the increased complexity of the system due to the integration of both systolic and shared-memory components. This complexity can lead to higher design and implementation costs, as well as increased power consumption. Additionally, the hybrid architecture may require specialized hardware extensions, such as Xqueue and Queue-linked registers, which could limit the portability of the system to other platforms. In terms of performance, the hybrid approach may not always outperform pure systolic or pure shared-memory architectures for specific workloads. Systolic arrays excel at regular dataflow patterns and can achieve high efficiency for certain types of computations. On the other hand, shared-memory clusters offer flexibility and programmability but may not be as efficient for highly parallel, compute-intensive tasks that benefit from systolic execution. The hybrid architecture may face challenges in optimizing the trade-offs between these two paradigms, potentially leading to suboptimal performance for certain types of workloads.

How could the hybrid architecture be leveraged to accelerate other types of workloads beyond the DSP kernels explored in the paper, such as irregular or data-dependent algorithms?

The hybrid architecture can be leveraged to accelerate a wide range of workloads beyond the DSP kernels explored in the paper, including irregular or data-dependent algorithms. One approach is to optimize the systolic topology and dataflow patterns to suit the specific requirements of these algorithms. For irregular algorithms, the systolic array can be dynamically reconfigured to adapt to the changing data dependencies and computation patterns. By utilizing the shared-memory cluster for flexible communication and data access, the hybrid architecture can efficiently handle irregular workloads that do not fit the traditional systolic model. Furthermore, the hybrid architecture can be extended with specialized accelerators or coprocessors to offload specific tasks or computations that are not well-suited for the systolic array or shared-memory cluster. By integrating domain-specific hardware components, the hybrid architecture can achieve higher performance and energy efficiency for a diverse set of workloads. Additionally, leveraging the programmability of the RISC-V cores in the shared-memory cluster allows for the implementation of custom algorithms and optimizations tailored to the specific requirements of irregular or data-dependent algorithms.