Deep Reinforcement Learning Based Scheduler for Flexible Job Shops with Sequence Dependent Setups
Year: 2020
Type: dissertation
Abstract: This thesis studies a flexible job shop scheduling problem (FJSP) with sequence dependent setups. FJSP becomes significantly complicated when there are several constraints related to re-entrant flows in manufacturing systems. At the same time, the scheduling problems need to be solved frequently to effectively manage the variabilities in production requirements, available machines, and initial setup status. Accordingly, scheduling decisions are usually required to be made on an hourly basis, making it challenging to obtain high quality schedules within the time limit for large-scale manufacturing systems. \n \nTo minimize the makespan of FJSP, three scheduling methods using deep reinforcement learning (DRL) are presented in this thesis. First, we suggest a decentralized scheduler (DS) in which each agent determines setup decisions in a decentralized manner and learns a policy by sharing a neural network among the agents to deal with the changes in the number of machines. Furthermore, novel definitions of state, action, and reward are proposed to address the variabilities in production requirements and initial setup status. \n \nSecond, we introduce a centralized scheduling approach in which an agent that determines actions given observations for jobs and machines. To reduce the complexity of state space inherent to the centralized learning, a novel definition of the state is developed by abstracting observations from an environment. Based on the centralized approach, we proposed two schedulers that select a rule-based method (CS-R) and a job-machine pair (CS-P) as an action, respectively. Specifically, CS-R employs the $Q$-network to learn a centralized policy and CS-P utilizes the actor-critic deep reinforcement learning method and Wolpertinger policy to approximate the continuous features of job-machine pairs. \n \nTo verify the robustness of the proposed method, neural networks (NNs) trained on small-scale scheduling problems are used to solve large-scale scheduling problems. Through extensive experiments on solving scheduling problems from real-world semiconductor packaging lines, we demonstrate that the proposed approach outperforms rule-based, meta-heuristic, and other RL methods in terms of the makespan while incurring shorter computation time than the meta-heuristics considered. Furthermore, the trained NN performs well in solving unseen real-world scale problems even under stochastic processing time, suggesting the viability of the proposed method for real-world manufacturing lines.
Author: 박인범
Language: English
Subfield: Industrial and Manufacturing Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3034954238