多核插值估计中的自适应隐式正则化与泛化分析

Large Adaptive Implicit Regularization and GeneralizationAnalysis for Multi-Kernel Interpolation

摘要: 核插值估计凭借隐式正则化带来的良性过拟合特性，已成为统计学习中突破经典偏差–方差权衡框架的重要研究方向。然而，单核函数难以适配复杂数据的多尺度特征，也无法自适应匹配核矩阵未知的特征值衰减规律，导致其在实际应用中性能受限。针对上述问题，本文将核插值的隐式正则化理论框架扩展至多核学习场景，提出一种自适应多核插值估计方法。该方法利用M 个正定内积核的线性组合构造多核函数，通过自适应优化核权重，克服单核插值在复杂数据表示与带宽选择上的局限，使模型具备自适应实现最优隐式正则化的能力。本文进一步建立了多核插值估计量积分平方风险的数据依赖型上界，并证明在高维情形下，该估计量以高概率满足泛化误差的收敛性。

Abstract: Kernel interpolation estimators have emerged as a prominent direction in statistical learning for overcoming the classical bias-variance trade-off, due to the benign overfitting enabled by implicit regularization. Nevertheless, single kernel functions are inadequate for capturing the multi-scale features in complex data and cannot adaptively match the unknown eigenvalue decay of the kernel matrix, which restricts their practical performance. To tackle these limitations, this paper extends the theoretical framework of implicit regularization for kernel interpolation to the setting of multiple kernel learning and proposes an adaptive multiple kernel interpolation estimator. The proposed estimator constructs a composite kernel as a linear combination of M positive definite inner-product kernels. By adaptively optimizing the kernel weights, it alleviates the drawbacks of single-kernel interpolation in representing complex data and selecting bandwidths, allowing the model to achieve optimal implicit regularization in a data-driven manner. Furthermore, we derive a data-dependent upper bound for the integrated squared risk of the multiple kernel interpolation estimator and prove that the estimator achieves convergence of the generalization error with high probability under high-dimensional settings.