Adaptive Networking and Resource Allocation for Distributed Data Parallel Training of Generative AI in Constrained Environments
Year: 2024
Type: dissertation
Abstract: As AI models and datasets grow larger, the demand for training resources has increased, leaving smaller companies with limited servers and no powerful GPUs at risk of falling behind. This study examines how small-scale environments can use Distributed Data Parallel (DDP) to participate in AI advancements, even with resource constraints. By deploying Docker containers across CPU-based servers connected via Ethernet, the experiment simulates low-resource conditions. Key network metrics are monitored, and a recommendation system is proposed for optimal node selection. The results show that distributed training is feasible in constrained environments, enabling smaller organizations to remain competitive in AI development.
Author: NAVADEHRAZI, OMID
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7054415138
Cites:
Cited by:
Related to: