Year: 2025
Type: dissertation
Abstract: This dissertation addresses the evolving challenges of deploying sophisticated artificial intelligence applications on resource-constrained edge devices. We present a comprehensive research trajectory that follows two parallel growth curves: increasing model complexity (from convolutional neural networks to transformers to Mixture-of-Experts architectures) and expanding edge device capabilities (from single low-power devices to powerful edge clusters). We begin with EdgeFlow, a distributed inference system for DAG-structured convolutional neural networks suitable for basic edge devices like Raspberry Pi. EdgeFlow's progressive partitioning algorithm transforms complex integer programming problems into more efficient linear programming formulations, reducing inference latency by up to 40\%. As we move to more sophisticated transformer models deployed on more capable edge devices, we introduce Voltage, which implements position-based partitioning specifically optimized for transformer architectures, reducing communication overhead by 4$\times$ compared to tensor parallelism while accelerating inference speed by up to 32.2\%. Addressing the increasing need for privacy preservation while training increasingly complex models, we develop \textsc{MaskCrypt} for federated learning environments. This system strategically encrypts only the most privacy-sensitive portions of model updates using a gradient-guided mask selection algorithm, reducing communication overhead by up to 4.15$\times$ while maintaining strong privacy guarantees against advanced attacks. As we progress to large language models requiring more powerful edge computing clusters, we present Menos, a memory-efficient split fine-tuning framework. Menos reduces GPU memory consumption by up to 72\% through spatial and temporal GPU memory sharing techniques, making privacy-preserving fine-tuning practical at scale even with limited edge resources. Finally, addressing the frontier of model complexity, we introduce VELA for Mixture-of-Experts models deployed on advanced edge clusters. By strategically distributing expert layers based on access frequency patterns, VELA reduces communication overhead by up to 25\% and accelerates the fine-tuning process by up to 28\%, making even the most sophisticated AI architectures viable at the edge. Through this systematic progression from simpler to more complex architectures across increasingly capable edge environments, this dissertation establishes a comprehensive framework for efficient AI deployment that scales with both model sophistication and available edge resources, enabling next-generation AI applications despite inherent constraints of edge computing.
Source: TSpace (University of Toronto)
Author: Hu, Chenghao
Institution: University of Toronto
Language: English
Topic: IoT and Edge/Fog Computing
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7132898468