Distributed Learning Techniques for Dynamic Access Control in massive Machine-Type Communication
Year: 2025
Type: dissertation
Abstract: Massive Machine-Type Communication (mMTC) plays a central role in supporting a vast amount of Internet of Things (IoT) devices for applications such as smart cities, smart factories, and environmental sensing.In fact, the number of connected devices is projected to reach 34.7 billion by 2028.Despite this growth, the availability of wireless resources is constrained by physical laws, and certain wireless frequency bands must be reserved for human communication needs.Therefore, limited number of wireless resources must be efficiently shared between an ever-increasing number of connecting devices.In this dissertation, we focus on improving multiple access control (MAC) protocols to solve the new scaling problem introduced by mMTC.Traditional centralized resource allocation schemes lead to large signalling overhead, while uncoordinated resource contention schemes unable to fully utilize the available bandwidth.We explore distributed reinforcement learning techniques to improve uncoordinated resource contention in mMTC and propose two tailored multi-agent learning methods for two branches of MAC.Our research components (RCs) are presented as follows:• RC#1 (TinyQMIX) TinyQMIX aims at dynamic channel selection under grant-free random access, leveraging distributed decision-making capability of multi-agent deep reinforcement learning to optimize channel usage.This work introduces a lightweight reinforcement learning algorithm designed for devices with limited computational power.TinyQMIX effectively reduces transmission delays by enabling devices to independently select optimal channels based solely on local observations.• RC#2 Mean-Field Transfer Thompson Sampling (MFTTS) MFTTS targeted at iv distributed power allocation in unsourced random access scenarios, by employing distributed online learning policies that based on Thompson Sampling (TS).Our proposed method, MFTTS, initializes local power selection models based on the convergence behavior of an approximated large-scale networks.We validated our propose methods using extensive simulations, which show that under particular mMTC scenarios, TinyQMIX and MFTTS outperform existing heuristic and reinforcement learning-based protocols in terms of delay reduction and power efficiency, respectively.v Acknowledgments First and foremost, I wish to express my deepest appreciation to Prof. Yusheng Ji for her unwavering support in both my professional research and personal life.She warmly welcomed me as an intern in her group during a particularly challenging time, allowing me to catch a last flight to Japan just before the borders closed during the peak of the COVID-19 pandemic in 2020.As her PhD student, I learned her tireless work ethic, "never give up" attitude
Authors: LE, Tien Thanh, レー, ティエン タイン, LE, Tien Thanh
Language: English
Topic: IoT Networks and Protocols
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Keywords: massive machine-type communications, dynamic access control, multiple access control, distributed reinforcement learning, multi-agent deep reinforcement learning, grant-free access, channel selection, unsourced random access, distributed power allocation, Thompson sampling, mean-field approximation, Internet of Things, resource contention, transmission delay reduction
Open Access status: green
OpenAlex ID: W7046808200
