无约束非凸优化高阶张量算法研究

Research on High-Order Tensor Algorithms for Unconstrained Non-Convex Optimization

摘要: 无约束优化问题在工程、经济及计算机视觉等领域领域有广泛应用，其目标函数因系统复杂与环境不确定呈高度非凸特性，使传统算法在效率与精度上双重受限。针对三次连续可微且三阶导数全局Lipschitz连续的非凸优化问题，现有方法局限显著：一阶算法(如SGD)收敛慢、易陷鞍点，难以达全局优化；二阶方法(如牛顿法)受Hessian矩阵存储计算开销限制，精确求解要求也降低适用性；三次正则化等三阶方法虽用高阶信息，但计算成本高，部分场景难构建全局光滑三阶逼近。文章基于信赖域范式与不动点迭代，结合目标函数高阶导数信息，通过减少迭代复杂度优化子问题求解，衔接理论与实际并减弱参数敏感性，设计了新型无约束非凸优化算法SSFPI-HOA。由数值实验可知，SSFPI-HOA算法能提升非凸问题最优解质量与计算效率，为领域提供新理论技术支撑，助力深度学习等领域实际问题求解。

Abstract: Unconstrained optimization problems play an important role in fields such as engineering, economics, and computer vision. Their objective functions exhibit highly non-convex characteristics due to system complexity and environmental uncertainty, rendering traditional algorithms doubly constrained in terms of efficiency and accuracy. For non-convex optimization problems that are three times continuously differentiable with globally Lipschitz continuous third-order derivatives, existing methods have significant limitations: first-order algorithms (e.g., SGD) converge slowly, are prone to getting trapped in saddle points, and hardly achieve global optimization; second-order methods (e.g., Newton’s method) are restricted by the storage and computational costs of Hessian matrices, and the requirement for accurate solution further reduces their applicability; third-order methods such as cubic regularization, although utilizing high-order information, incur high computational costs and struggle to construct globally smooth third-order approximations of objective functions in some scenarios. Based on the trust region paradigm and fixed-point iteration, this study combines the high-order derivative information of objective functions and designs a novel unconstrained non-convex optimization algorithm (SSFPI-HOA) by reducing iteration complexity, optimizing subproblem solving, bridging theory and practice, and reducing parameter sensitivity. Numerical experiments show that the SSFPI-HOA algorithm can improve the quality of optimal solutions and computational efficiency for non-convex problems, providing new theoretical and technical support for the field and facilitating the solution of practical problems in areas such as deep learning.