Year: 2026
Type: dissertation
Abstract: Efficient transformer inference on edge hardware requires optimization of both model precision and memory utilization.Transformer models are often memory-bound, which makes quantization especially important for reducing memory footprint and data movement.Lower-precision integer formats can improve deployability on embedded devices, but they may also introduce accuracy loss when applied to sensitive operations or tensors with large outliers.The study focuses on mixed-precision quantization and rotation-based methods, evaluated using vision transformer models deployed on Arm Ethos-U hardware accelerators.Mixed precision is used to preserve accuracy in sensitive parts of the model, while rotations are used to improve weight and activation quantization.The methods are compared using accuracy, memory usage, and hardwareoriented performance metrics.The results confirm that reducing precision has a positive effect on memory and performance, but that mixed-precision is essential in some cases in order to avoid complete model degradation.Rotations can help recover part of the accuracy-efficiency trade-off introduced by lower precision, especially for weights, but effectiveness depends on model architecture.
Authors: Holmquist, Märta, Kujala, Emma
Institution: Lund University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166495294
