TP-DCIM: Transposable Digital SRAM CIM Architecture for Energy-Efficient and High Throughput Transformer Acceleration
Year: 2024
Type: conference-paper
Abstract: To accelerate the execution of transformer models, compute-in-memory (CIM) has been widely adopted. However, the CIM architecture has the drawback of fixed one-way computing structure supporting only horizontal input sharing vertical accumulation (HIVA). So, it faces two major obstacles: 1) Matrix transposition processing needs large hardware overheads, 2) A fixed dataflow incurs CIM underutilization problem, degrading throughput. In this paper, we present a digital SRAM CIM (DCIM)-based transformer accelerator that supports two-way computing: HIVA and vertical input sharing horizontal accumulation (VIHA). The proposed two-way computing DCIM macro features a novel 9T SRAM bitcell and transposable adder tree, which efficiently reduces matrix transposition costs. We also present a novel dataflow to improve overall self-attention latency by resolving CIM underutilization and hiding dynamic input generation latency. In addition, CIM-friendly computation skipping scheme is exploited to enhance energy efficiency with negligible accuracy loss. The simulation results show that the proposed DCIM-based accelerator achieves up to 2.5x latency improvement and 24% energy savings compared to previous CIM-based accelerators.
Authors: Junwoo Park, Kyeongho Lee, Jongsun Park
Institution: Korea University
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4409282663