基于时延Q学习的机器人动态规划方法

Dynamic Planning Method Based on Time Delayed Q-Learning

摘要: 主要针对现有机器人动态规划方法环境未知，且收敛性能欠佳的缺点，提出了一种基于时延Q学习的机器人动态规划方法。首先，对机器人规划进行了MDP建模，将其转换为一个可以通过强化学习解决的问题。然后，定义了规划的目标函数，并描述了基于时延Q学习的机器人规划算法。在该算法中采用Rmax方法来初始化所有状态动作对的Q值，使得所有状态动作对都能被探索到，同时通过时延的Q值来减少Q值更新的次数，从而提高Q值更新的效率。仿真实验表明：文中设计的时延Q学习算法能有效地实现移动机器人的路径规划，较其它算法相比，具有收敛效果好和收敛速度快的优点，具有较大的优越性，是一种有效的机器人动态规划方法。

Abstract: Aiming at the unknown environment of the existing robot dynamic planning methods with the slow convergence, a robot planning method based on time delayed Q-Learning is proposed. Firstly, the robot planning is modeled as MDP model, and it is then transferred as the problem which can be solved by reinforcement learning method. Then, the goal function of dynamic planning is defined, and the planning algorithm based on time delayed Q-Learning is proposed. The Q value of every state action pair is initialized to Rmax, so that all the state action pairs are explored, via decreasing the number of updating for Q value, to improve the updating efficiency. The simulation experiment shows: this time delayed Q-Learning algorithm can achieve the path planning of the mobile robot; compared with the other methods, this method has the advantages of good convergence performance and quick convergence rate with big priority, thus it is an effective robot planning method.