Year: 2026
Type: dissertation
Abstract: This thesis investigates the memory dynamics of neural computing models, with a focus on balancing learning stability and memory efficiency. First, it addresses memory stability by analyzing catastrophic forgetting in Kolmogorov-Arnold Networks (KANs) and introduces KAN-LoRA, a novel adapter architecture that enables incremental knowledge updates in large language models while minimizing interference. Second, it examines memory efficiency by establishing communication lower bounds for matrix sketching and Nyström low-rank approximation in distributed-memory environments and by proposing communication-optimal parallel algorithms. These algorithms, implemented on modern CPU and GPU supercomputing infrastructures, achieve substantial parallel scalability and significantly reduce the communication overhead in large-scale matrix computations. Building on these insights, the thesis further presents a data-aware extreme compression algorithm for large language models, preserving targeted knowledge in the compressed model and facilitating the deployment of advanced AI on resource-constrained systems. Overall, this work advances the understanding of neural memory dynamics and provides practical methods for stable, efficient, and scalable neural computation.
Author: Rahman, Mohammad Marufur Marufur
Institution: Canadian Generic Pharmaceutical Association
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7169646405