海量数据中的分布式支持向量回归

Distributed Support Vector Regression of Massive Datasets

摘要: 大规模的数据给传统的统计推断方法带来了新的挑战，比如在分析超过一台计算机容量的海量数据集时，由于数据太大，无法保存在计算机内存中，计算任务可能需要花费很长时间才能获得结果。为了有效地解决海量数据情形下的诸多问题，本文研究了支持向量回归的分布式估计。首先采用平滑技术发展了一种平滑支持向量回归(S-SVR)估计方法。然后基于分而治之的思想，针对海量数据集对S-SVR估计方法提出了分而治之支持向量回归估计算法(DC-SVR)，该方法解决了内存限制和计算时间的问题。此外，本文中提出的DC-SVR方法中的参数可通过网格搜索和交叉验证相结合的方法获得，具有自适应性，其中最优的参数是由每次数据自动选择的。在模拟研究中，通过不同情形的实验表明了文章所提估计量的优越性，模拟结果显示通过DC-SVR所得的估计量在平均绝对偏差和均方误差评价准则下的差异更小。

Abstract: Large scale data brings new challenges to the traditional statistical inference methods. For example, when analyzing massive datasets whose sizes usually exceed the capacity of a single computer, the data is too large to be saved in computer memory and computing task may take too long to get the results. In order to effectively solve many problems in the case of massive data, this paper studies the distributed estimation of support vector regression. Firstly, we develop smoothed support vector regression (S-SVR) estimation method. Then based on the idea of divide-and-conquer, we propose divide-and-conquer support vector regression estimation algorithm (DC-SVR) for the S-SVR estimation method of massive datasets. This method solves the problems of memory limitation and computing time. In addition, the parameters in the DC-SVR method can be obtained by the combination of grid search and cross validation, which is adaptive. The optimal parameters are automatically selected by each data. In the simulation study, a lot of numerical simulation studies are carried out to verify the superiority of our proposed distributed estimator. The simulation results show that the DC-SVR estimator has less difference under the evaluation criteria of mean absolute deviation and mean square error.