Type: dissertation
Abstract: AI and especially large language models, face critical deployment challenges as inference con- sumes substantially more energy than training over their operational lifecycle. This thesis investigates efficient inference methodologies through three complementary approaches. First, we explore neural network scaling laws and training-inference compute trade-offs, exploring how Chinchilla scaling laws can be adapted to account for inference costs. Second, we analyze com- putational patterns in transformer architectures to understand optimization targets. Third, we conduct empirical studies comparing quantized and full-precision models across multiple scales, demonstrating that certain quantization methods and techniques maintain performance while reducing computational requirements. We extend those studies by examining test-time scaling behaviors of model, seeing if smaller models can more usefull than larger models if given extra compute at inference time. Our comprehensive analysis of weight and activation quantization establishes quantization as a fundamental technique for efficient deployment. This research provides both theoretical frameworks and practical methodologies for optimizing in- ference operations, enabling sophisticated language model deployment in resource-constrained environments while maintaining high performance.
Author: ŠUBIĆ, TOMISLAV
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7221301322