基于改进梯度下降算法的XGBoost模型研究

Research on XGBoost Model Based on Improved Gradient Descent Algorithm

摘要: 分数阶微积分的优良性质为基于梯度下降的优化方法提供了新的前景，相比于传统整数阶，分数阶微积分可以通过不断更新来调整阶数。XGBoost模型结合了正则化技术和梯度提升决策树，具有高效性和准确性。但在实际应用中，XGBoost模型仍存在训练速度较慢、易过拟合等局限性。为了解决这些问题，本文所做工作主要有如下几个方面：1) 基于Caputo定义引入了分数阶微积分的基本内容。作为一种经典的凸优化算法，梯度下降法在结合分数阶微积分理论后延伸出了分数阶梯度下降算法。2) 在简化目标函数的过程中，将分数阶梯度下降算法与XGBoost模型相结合，并选择合适的分数阶，得到新的分数阶XGBoost模型，在保证准确性的同时提高了训练速度。3) 利用改进后的XGBoost模型对房价预测数据集进行实证分析。与传统的整数阶XGBoost模型进行对比分析，改进后的模型具有更快的收敛速度和更好的预测性能。

Abstract: The excellent properties of fractional calculus provide a new prospect for optimization methods based on gradient descent, which can be continuously updated to adjust the order compared to the traditional integer order. The XGBoost model combines regularization techniques and gradient boosting decision trees, which is efficient and accurate. However, in practical applications, the XGBoost model still has limitations such as slow training speed and easy overfitting. In order to solve these problems, the work done in this paper is mainly as follows: 1) Based on the Caputo definition, the basic content of fractional calculus is introduced. As a classical convex optimization algorithm, the gradient descent method is extended to the fractional gradient descent algorithm after combining the fractional calculus theory. 2) In the process of simplifying the objective function, the fractional gradient descent algorithm is combined with the XGBoost model, and the appropriate fractional order is selected to obtain a new fractional XGBoost model, which improves the training speed while ensuring the accuracy. 3) The improved XGBoost model is used to conduct an empirical analysis of the housing price prediction dataset. Compared with the traditional integer order XGBoost model, the improved model has faster convergence speed and better prediction performance.