基于双Q学习算法的可重入混合流水晶圆车间调度与维护联合优化研究

Research on Joint Optimization of Scheduling and Maintenance of Crystal Circle Shop for Re-Entrant Hybrid Flow Based on Double Q Learning Algorithm

摘要: 针对可重入晶圆车间调度与预维护问题的复杂性，以及近年来人工智能算法的飞速发展和启发式算法在优化复杂生产系统上的不足，本文提出了一种基于双Q学习的可重入晶圆车间调度与维护联合优化模型。首先，考虑到可重入工序的影响，在调度阶段建立可重入调度约束模型。考虑可重入工序以及设备维护后的役龄更新对设备维护频次的影响，并通过设立维护阈值来对设备进行预维护以及机会维护，建立考虑役龄更新的可重入晶圆车间设备维护模型。其次，结合实际生产系统中多因素的影响，以最小完工时间、最低能源消耗以及最低总维护成本为目标函数进行多目标优化。最后，以生产数据为基础，通过双Q学习算法来定义状态和动作，并设置建立奖励函数，采用贪婪策略随机选择动作来跳出局部最优，并通过调整役龄更新因子来进行灵敏度分析验证算法的鲁棒性。经过结果分析以及对比分析，基于双Q学习算法所建立模型的结果均取得了较好的优化结果，并且具有较强的鲁棒性，证明了所提出的基于双Q学习算法的可重入晶圆车间调度与维护联合优化模型的有效性。

Abstract: Due to the complexity of reentrant wafer workshop scheduling and preventive maintenance issues, as well as the rapid development of artificial intelligence algorithms in recent years and the limitations of heuristic algorithms in optimizing complex production systems, this paper proposes a reentrant wafer shop scheduling and maintenance joint optimization model based on double-Q learning. First, considering the influence of reentrant processes, a reentrant scheduling constraint model is established in the scheduling stage. Considering the influence of reentrant process and service age update after equipment maintenance on equipment maintenance frequency, and setting up maintenance thresholds for pre-maintenance and opportunity maintenance of equipment, a reentrant wafer shop equipment maintenance model considering service age update is established. Secondly, the minimum completion time, minimum energy consumption and minimum total maintenance cost are taken as the objective functions for multi-objective optimization, taking into account the influence of multiple factors in the actual production system. Finally, based on the production data, the double-Q learning algorithm is used to define the states and actions, and the reward function is set up to establish a greedy strategy to randomly select the actions to jump out of the local optimum, and the sensitivity analysis is carried out by adjusting the service age update factor to verify the robustness of the algorithm. After the result analysis and comparative analysis, the results have achieved better optimization results and have strong robustness, which proves the effectiveness of the proposed re-entry wafer shop scheduling and maintenance joint model.