Work

Quantization Techniques for Transformer Models on Ethos-U Hardware Accelerators

Year: 2026
Type: dissertation
Abstract: Efficient transformer inference on edge hardware requires optimization of both model precision and memory utilization.Transformer models are often memory-bound, which makes quantization especially important for reducing memory footprint and data movement.Lower-precision integer formats can improve deployability on embedded devices, but they may also introduce accuracy loss when applied to sensitive operations or tensors with large outliers.The study focuses on mixed-precision quantization and rotation-based methods, evaluated using vision transformer models deployed on Arm Ethos-U hardware accelerators.Mixed precision is used to preserve accuracy in sensitive parts of the model, while rotations are used to improve weight and activation quantization.The methods are compared using accuracy, memory usage, and hardwareoriented performance metrics.The results confirm that reducing precision has a positive effect on memory and performance, but that mixed-precision is essential in some cases in order to avoid complete model degradation.Rotations can help recover part of the accuracy-efficiency trade-off introduced by lower precision, especially for weights, but effectiveness depends on model architecture.
Institution: Lund University
Language: English
Open Access status: green
OpenAlex ID: W7166495294
Cites: 30
Cited by:
Related to: 10