Advancing Hyperdimensional Computing Based on Trainable Encoding and Adaptive Training for Efficient and Accurate Learning
Year: 2024
Type: article
Abstract: Hyperdimensional computing (HDC) is a computing paradigm inspired by the mechanisms of human memory, characterizing data through high-dimensional vector representations, known as hypervectors. Recent advancements in HDC have explored its potential as a learning model, leveraging its straightforward arithmetic and high efficiency. The traditional HDC frameworks are hampered by two primary static elements: randomly generated encoders and fixed learning rates. These static components significantly limit model adaptability and accuracy. The static, randomly generated encoders, while ensuring high-dimensional representation, fail to adapt to evolving data relationships, thereby constraining the model’s ability to accurately capture and learn from complex patterns. Similarly, the fixed nature of the learning rate does not account for the varying needs of the training process over time, hindering efficient convergence and optimal performance. This article introducesTrainableHD, a novel HDC framework that enables dynamic training of the randomly generated encoder depending on the feedback of the learning data, thereby addressing the static nature of conventional HDC encoders.TrainableHDalso enhances the training performance by incorporating adaptive optimizer algorithms in learning the hypervectors. We further refineTrainableHDwith effective quantization to enhance efficiency, allowing the execution of the inference phase in low-precision accelerators. Our evaluations demonstrate thatTrainableHDsignificantly improves HDC accuracy by up to 27.99% (averaging 7.02%) without additional computational costs during inference, achieving a performance level comparable to state-of-the-art deep learning models. Furthermore,TrainableHDis optimized for execution speed and energy efficiency. Compared to deep learning on a low-power GPU platform like NVIDIA Jetson Xavier,TrainableHDis 56.4 times faster and 73 times more energy efficient. This efficiency is further augmented through the use of Encoder Interval Training (EIT) and adaptive optimizer algorithms, enhancing the training process without compromising the model’s accuracy.
DOI: 10.1145/3665891
Authors: Jiseung Kim, Hyunsei Lee, Mohsen Imani, Yeseong Kim
Language: English
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Funders: National Science Foundation, Semiconductor Research Corporation, Cisco Systems, National Research Foundation of Korea, Ministry of Science and ICT, South Korea, Defense Advanced Research Projects Agency, Institute for Information and Communications Technology Promotion, Office of Naval Research, Air Force Office of Scientific Research
Awards: Neurally-Inspired Integration of Communication and Cognitive Computation in Hyperspace, G2871076685, G2874769152, G2976525445, G3908657300, UKRI/BBSRC-NSF/BIO: Interpretable and Noise-Robust Machine Learning for Neurophysiology, G4700831490, G5553233121, CPS: Small: Brain-Inspired Memorization and Attention for Intelligent Sensing, G6981114264, G7449898391, G8017170229, G8824570850, Hyperdimensional Neural Computation for Real-Time Cognitive Learning
OpenAlex ID: W4399321415
