Year: 2025
Type: dissertation
Abstract: Τhis diploma thesis investigates the challenges and opportunities of Small Language Models (SLMs) as efficient, privacy-aware alternatives to Large Language Models (LLMs) in resource-constrained and real-time environments.The work combines a systematic literature review of architectural and optimization techniques for SLMs with the design and implementation of a practical prototype that integrates fine-tuning, Retrieval-Augmented Generation (RAG), and model quantization for multi-platform deployment (desktop and mobile).Methodologically, the thesis surveys state-of-the-art SLM frameworks and compression strategies (pruning, distillation, quantization), selects and fine-tunes Gemma family models under constrained hardware, and evaluates the system using standard automatic metrics (token-level F1, BLEU-1, ROUGE, BERTScore) together with deployment-oriented measurements (latency, memory footprint).The implementation explores both full fine-tuning and parameter-efficient LoRA adaptation, multiple quantization formats, a multi-source retrieval pipeline, and an offline mobile setup.Key findings show that the combined strategy of task-specific fine-tuning plus RAG substantially improves answer accuracy and stability, enabling compact models to approach the utility of larger systems for domain tasks.Larger SLM variants (e.g., Gemma-3-4B) consistently outperform smaller ones (e.g., Gemma-3-1B) under otherwise similar training regimes, while conservative quantization of larger architectures provides an attractive costperformance trade-off; by contrast, aggressive quantization harms the smallest models' recall and semantic fidelity.The thesis demonstrates practical on-device deployment (PocketPal; Gemma-1B q8_0 on an iPhone 12 with 4 GB RAM) as a viable privacy-preserving option.Overall, the work argues that SLMs-when combined with careful data curation, RAG, and hardware-aware compression-can democratize generative AI by lowering cost and energy barriers, enabling offline/private applications, and offering competitive performance Institutional Repository -Library & Information Centre -University of Thessaly 25
Author: Κοτρώτσιος, Χρήστος Β.
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7223537503