Year: 2021
Type: dissertation
Abstract: Deep neural networks have enabled numerous AI applications in various fields, obtaining higher accuracy in many tasks than conventional heuristic approaches.This great success of deep neural networks was attained by advances in hardware technology typified by GPUs and the invention of model structure and training algorithms.Dedicated hardware accelerators for neural networks on FPGAs and ASICs have been proposed to compute a neural network model in acceptable latency and throughput.On the other hand, the algorithms are continuously evolving, getting more complex and massive; hardware-oriented algorithms to reduce the computational complexity or amount have also been proposed.We expect that hardware-software coevolution drives further development of AI-enabled applications and establishes the next generation of information processing with AI-native computing power.This thesis introduces programmable neural network processors for edge-side inference and accurate quantized neural network algorithms for embedded systems, with the concepts appropriately verified by FPGA and ASIC prototyping.Through this, we show a practical example and of hardware-software co-optimization, which proves its effectiveness by quantitative evaluation.First, we prototype a reconfigurable parallel processor for convolutional neural networks.A convolutional neural network consists of two types of computational layers, convolution (Conv) and fully-connected (FC) layers, which have different computational and data structures.We extract a one-to-all parallel multiply-accumulation (MAC) as the common primitive operation in both Conv and FC layers.We construct an ideal parallel computational array with shared and individual buses to conduct this primitive operation.After that, we make this architecture feasible by relaxing internal data rates with an in-array data sharing mechanism and multithreaded accumulators.We discuss the requirements for hardware and model structures for better efficiency through the architecture exploration using this architecture.We then propose a near-memory processor, named BRein Memory, which improves neural network processing efficiency by eliminating external data movements.It is known that the energy and latency of external memory access set limitations on the performance and efficiency of neural network processing.We attempt to omit the external memory throughout the entire processing of a neural network model.We adopt the binary neural network algorithm, where all the activation and weights are restricted to be -1/+1.It allows the multiplication to be calculated as bitwise XNOR.Thanks to this light-weight arithmetic, we integrate thin processing units between two SRAM macros to close the parallel computation and data movement in it.The computational units can be cascaded to compose a pipeline by matching the symmetric parallelisms in a neural network layer with the SRAM data access pattern, where no scratchpad memory is required.We fabricated a prototype LSI of this architecture with six processing blocks, which can house a 13-layer neural network model at most, achieving 2.3 TOPS/W energy efficiency, the best figure in neural network processor ASICs at the time published.The concept of highly efficient near-memory processing with a quantization algorithm has been proven by prototyping and evaluating this architecture.Quantized neural networks reduce the computational complexity and memory footprint.However, the accuracy drop is unavoidable, and this could be a fatal problem in some situations.Here, we discuss an accurate yet efficient quantization algorithm that can be used in compact hardware implementation.A neural network processor is a digital signal processor when we see the computation carried in the processing units.The knowledge in signal processing gives us a hint for improving the accuracy of quantized neural networks on compact hardware.Dithering is a technique for low-precision quantization that reduces the quantization errors by representing the source data in a stochastically and spatially distributed manner.Its simplest algorithm is error diffusion, which can be computed by adding each pixel's quantization error to the neighbor.A processing unit in a neural network processor has an adder, which can be appropriated for quantization error accumulation.Based on this idea, we propose a quantization algorithm with error diffusion named Dither NN.Since we can use the adders that the neural network processor has, no additional arithmetic units are required for this extension.We implemented prototype architectures on an FPGA, and we trained convolutional neural network models with and without dithering.We proved the accuracy improvements with a very few additional hardware resource occupation through the evaluation.This thesis provides a systematic methodology of architecture construction and a practical example of hardware-algorithm co-optimization, focusing on the computation primitives and data delivery patterns through these discussions and prototyping.The key LIST OF FIGURES v 5.10 Typical procedure of the MAC operation of the fixed-point activations and weights.5.11 Dithering operation in the fixed-point computation.5.12 Dithering operation in the fixed-point computation when the truncation is rounding toward zero.5.13 Typical procedure of the MAC operation of the logarithmic activations and weights.5.14 Dithering support in a typical logarithmic PE. List of Tables3.1 Processing time [cycles] and data rate [bps] 3.2 Evaluation of the proposed architecture using AlexNet 3.3 Energy comparison of multi-port and multi-bank implementations 4.1 Comparison on the same 13-layer binary DNN (measured) 4.2 Comparison and chip summary 4.3 Layer Configurations of the Experimental MNIST CNN 5.1 Test network model architectures A 5.2 Test network model architectures B: MobileNetV1 5.3 Implementation result of the prototype architectures vi
Authors: 安藤, 洸太, Ando, Kota
Institution: Tokyo Institute of Technology
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7054031259