Year: 2023
Type: dissertation
Abstract: Massive machine-type communication is one of the three use cases of 5th generation (5G) and beyond wireless communications technology, which enables connectivity of a large number of short-packet internet-of-things (IoT) devices. The Third Generation Partnership Project specifies IoT devices’ characteristics, such as asynchronous network access (sporadically accessing the network), and low-cost devices with minimal power consumption and low data rates, especially uplink. Such specific characteristics mandate the development of new access and connectivity methods for massive and critical IoT communications with special attention to 1) massive connectivity for short-packet communications, 2) low complexity with less overhead control signalling, 3) energy constrained and uplink-dominated networks, 4) heterogeneous quality of service (QoS) requirements, 5) channel utilization and ultra-high spectrum efficiency. In view of such expectations, non-orthogonal multiple access (NOMA) is considered a promising solution to satisfy massive connectivity demands and provide high spectral efficiency for future wireless communication. NOMA has the potential to serve multiple users on the same time-frequency resource block (RB) with different transmit power levels. Cellular IoT networks with NOMA use two types of random-access (RA) protocols, i.e., grant-based (GB) and grant-free (GF) access. GB access fails to provide sufficient access to IoT users with short packets because multiple handshakes are required before the transmission. Therefore, GF-NOMA which eliminates the handshaking process is proposed for communication with sporadic traffic because of its arrive-and-go manner. GF-NOMA is suitable for low-priority IoT users (e.g. sensors for environmental monitoring); however, it may not provide a stable connection for high-priority users (such as sensors for remote health monitoring) demanding stringent QoS requirements. Recently, another RA-NOMA scheme, namely semi-grant-free (SGF)-NOMA has been proposed to guarantee users’ heterogeneous QoS requirements and enhance the connectivity, where one GB user shares the redundant resources with several other GF users. Since NOMA exploits the power domain for multiple access, power allocation is vital to achieve superior spectral efficiency and further enhance network performance. Effective power control algorithms can improve communication quality, reduce interference, and save energy. Machine learning, specifically reinforcement learning (RL) algorithms can learn from data and optimize power control policies that improve the performance of wireless communication systems. This thesis provides ML-based power control for the aforementioned RA methods to address complexity at BSs, QoS requirements of different users, signalling overhead, scalability and users’ collision problems. First, we designed a transmit power pool (PP) associated with users’ location information for GF-NOMA IoT networks via cooperative multi-agent deep RL (MA-DRL). The PP contained several optimal transmit power levels, and the goal of this transmit PP design was to increase the efficiency of power usage, reduce interference, and improve overall network performance. Data transmitting IoT users acquire their transmit power in advance from this PP according to their communication distances to avoid the exchange of channel state information, which reduces signal overhead and complexity at BS. Comparing with the conventional online learning approach, the proposed algorithm with the PP converges faster under changing environments by limiting the action space based on the previous learning. In terms of throughput, the considered GF-NOMA system outperforms the networks with fixed transmission power and the traditional GF with orthogonal multiple access techniques. Second, we changed the PP design from location orientated to RB-orientated to ensure users’ heterogeneous QoS requirements are met. In particular, we designed RB-oriented PPs in the presence of residual errors resulting from imperfect successive interference cancellation, which enables distributed open-loop power control. We demonstrated that the proposed algorithm is computationally scalable to large-scale IoT networks and produces minimal signalling overhead. The proposed scheme outperforms the existing SGFNOMA system and networks with pure GF protocols. We have shown the impact of the GB user shares the redundant resources with several other GF users. Since NOMA exploits the power domain for multiple access, power allocation is vital to achieve superior spectral efficiency and further enhance network performance. Effective power control algorithms can improve communication quality, reduce interference, and save energy. Machine learning, specifically reinforcement learning (RL) algorithms can learn from data and optimize power control policies that improve the performance of wireless communication systems. This thesis provides ML-based power control for the aforementioned RA methods to address complexity at BSs, QoS requirements of different users, signalling overhead, scalability and users’ collision problems. First, we designed a transmit power pool (PP) associated with users’ location information for GF-NOMA IoT networks via cooperative multi-agent deep RL (MA-DRL). The PP contained several optimal transmit power levels, and the goal of this transmit PP design was to increase the efficiency of power usage, reduce interference, and improve overall network performance. Data transmitting IoT users acquire their transmit power in advance from this PP according to their communication distances to avoid the exchange of channel state information, which reduces signal overhead and complexity at BS. Comparing with the conventional online learning approach, the proposed algorithm with the PP converges faster under changing environments by limiting the action space based on the previous learning. In terms of throughput, the considered GF-NOMA system outperforms the networks with fixed transmission power and the traditional GF with orthogonal multiple access techniques. Second, we changed the PP design from location orientated to RB-orientated to ensure users’ heterogeneous QoS requirements are met. In particular, we designed RB-oriented PPs in the presence of residual errors resulting from imperfect successive interference cancellation, which enables distributed open-loop power control. We demonstrated that the proposed algorithm is computationally scalable to large-scale IoT networks and produces minimal signalling overhead. The proposed scheme outperforms the existing SGFNOMA system and networks with pure GF protocols. We have shown the impact of the level of SIC error on the sum rate and average user’s transmit power. Simulation results show that the PP can benefit new (untrained) users joining the network and outperform conventional SGF-NOMA without PPs in terms of spectral efficiency. Third, to ensure efficient and reliable GF-NOMA and SGF-NOMA operation, we focused on the power collision problem. GF users in GF-NOMA and SGF-NOMA share a common pool of power levels; a power collision occurs when one user’s power is allocated to another user on the same RB. To solve the collision problem and maximize conditional throughput, we developed an efficient MA-DRL framework for optimizing PP and access strategies. In particular, this work exploits the fast-response feature of the traditional competitive MA-DRL and the increased-performance feature of the traditional cooperative MA-DRL to redesign a mixed reward system that contributes to a hybrid MA-DRL mode for enhancing agents’ learning efficiency. The proposed hybrid MA-DRL at the cluster level converges faster to an optimal solution than that at the network level, but it does so at the cost of user clustering. As a result, the hybrid scheme increases the number of successful decoded users compared to traditional schemes without learning capability. The proposed hybrid MA-DRL mode performs better than the pure competitive and cooperative MA-DRL modes, especially under a heavy-load network.
Author: Fayaz, M
Institution: Queen Mary University of London
Language: English
Topic: IoT Networks and Protocols
Subfield: Electrical and Electronic Engineering
Field: Engineering
Domain: Physical Sciences
Keywords: noma, random access protocols, power control, transmit power allocation, reinforcement learning, deep reinforcement learning, deep Q-network, deep deterministic policy gradient, short packet transmission, massive machine-type communications, Internet of Things, uplink power control, energy efficiency
Open Access status: green
OpenAlex ID: W7058032364
