Year: 2026
Type: dissertation
Abstract: Reducing numerical precision is a promising approach to lowering the computational and memory requirements of neural network training.This dissertation investigates low-precision training through a structured literature review, followed by an experimental comparison of selected frameworks with publicly available implementations identified during the review.The review analyzes thirteen studies, examining the adaptations introduced to conventional neural network training, numerical representations and precision configurations used across different stages of the training process.It further analyzes the predictive performance, computational efficiency and resource usage, identifying the main trade-offs and evaluation differences.One of the main gaps identified in the review is the lack of a consistent experimental setting for comparing low-precision training approaches.Existing studies use different datasets, network architectures, hardware, training configurations and efficiency metrics, making direct comparison difficult.To address this, the experimental study developed in this dissertation compares NITI, NITRO-D and PocketNN with an FP32 baseline under partially harmonized conditions, using common datasets, architectures and evaluation metrics whenever possible.The experiments include multilayer perceptrons on MNIST and CIFAR-10 datasets and convolutional neural networks on CIFAR-10, considering predictive performance, execution time, throughput, memory usage and, for the convolutional experiments, device-level GPU energy.Results show that the evaluated low-precision approaches can maintain predictive performance close to FP32 in some configurations, but do not provide consistent reductions in execution time or memory usage.Across the final comparisons, the FP32 baseline achieved the highest mean final test accuracy and the shortest execution time.In the common-architecture convolutional comparison, FP32, NITI and NITRO-D achieved mean final test accuracies of 85.843%, 84.391% and 83.141%, respectively, while NITI and NITRO-D required approximately 1.92 and 6.71 times the FP32 execution time.Memory usage varied according to the framework, dataset and type of memory considered.Overall, reducing numerical precision alone does not guarantee lower practical resource requirements.The observed trade-offs depend on the complete training approach and experimental setting, including the learning algorithm, software implementation, training configuration and hardware used.This dissertation therefore provides a comparative view of existing low-precision training approaches and experimental evidence on the capabilities and limitations of selected publicly available frameworks, while identifying directions for further research.
Author: Mariana Solange Monteiro Rocha
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225352796