Work

Vahvistusoppimisen soveltaminen fyysisen kerroksen töiden aikatauluttamiseen

Year: 2024
Type: dissertation
Abstract: Societal demand for higher connectivity leads to the continuous increase of requirements for wireless communication. There is a need to seek new solutions to meet this demand. Although 5G New Radio has been a reliable method to tackle this predicted demand and envisioned use cases for wireless communication, there is ongoing research for more advanced solutions for 6G. One aspect is to include Machine Learning (ML) as part of the Physical Layer (L1) since ML solutions excel in such a stochastical environment. Analysis of L1 job scheduling has been introduced in research. Using ML algorithms in L1 job scheduling is an unexplored topic since there are no publicly available data sources on which to train. Job scheduling is often an NP-hard combinatorial optimization problem, and finding an optimal solution is complex. Therefore, it is advantageous to use Reinforcement Learning (RL) algorithms for such problems. This work studied how scheduling data from a 5G PUSCH uplink link-level simulator and standardized scheduling instances can be used to train state-of-the-art RL algorithms. Furthermore, trained algorithms were compared to classical scheduling ones in solving NP-hard job scheduling problems. An end-to-end system from the 5G PUSCH uplink data source to reinforcement learning training and testing was designed and implemented to achieve these goals. The performance of the trained RL algorithms is evaluated using a standardized makespan metric. The results show that the developed system enabled the training of RL algorithms that could outperform classical scheduling algorithms in selected benchmarks. In general, the top-performing algorithms achieved a 3 to 15 percentage point reduction in makespan compared to the widely used FIFO algorithm. The Proximal Policy Optimization (PPO) algorithm performed consistently in every benchmark. Even with small neural networks, the PPO excels in the 5G PUSCH uplink scheduling scenarios, lowering the computational requirements to run inference. The trained PPO fell around 11 percentage points behind the state-of-the-art results observed in the literature. Multiple methods were identified that could improve the obtained results.
Author: Eronen, Ville
Language: English
Open Access status: green
OpenAlex ID: W6997317314
Cites: 50
Cited by:
Related to: 10