Year: 2022
Type: conference-paper
Abstract: Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively.
Institution: University of California, Santa Barbara
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Awards: G1875027798, G2045646697, G325663671, G3319042522, G4228342300, G4250104368, G445623813, CC* Compute: A high-performance GPU cluster for accelerated research, G6512159266, Materials Research Science and Engineering Center at UCSB, G7061003935, G7437795455, G8230913862, FMitF: Track I: Scalable and Quantitative Verification for Neural Network Analysis and Design
OpenAlex ID: W4214686755
