稀疏模型下的模型选择方法比较及应用

Application and Comparison of Model Selection Methods under Sparse Models

摘要: 模型选择是统计研究的热点，随着大数据时代的到来，数据维数越来越高，在经济金融学领域，生物统计领域，图像处理等领域对模型的选择产生了更大的需求。同时稀疏模型在机器学习中发挥越来越重要的作用，它可以避免模型过度拟合的情况。本文主要基于多元线性回归模型进行模型选择的研究，主要对带有惩罚因子的模型选择方法中的岭回归、LASSO和SCAD三种方法和贝叶斯模型选择方法进行了总结阐述。后面通过数据模拟和实例分析，重点研究在稀疏性模型的前提下，对这四种模型选择的方法进行分析对比。通过研究分析可以发现在模型稀疏性较强的情况下，岭回归表现较好，SCAD方法能较好地去除不重要变量，结果与岭回归相差甚小，由此可以根据这种良好的性质在实际的应用当中得到充分利用。

Abstract: In this paper, we study the model selection in multiple linear regressions. Model selection is a hot topic in statistical research. With the advent of the era of large data, the dimension of data is getting higher and higher. There is a greater demand for model selection in the fields of economics and finance, biostatistics and image processing. At the same time, sparse model plays an increasingly important role in machine learning, which can avoid over-fitting. This paper mainly studies the model selection based on multiple linear regression model, and summarizes the ridge regression, LASSO and SCAD methods and Bayesian model selection methods. Later, through data simulation and case analysis, we focus on the sparse model under the premise of the four model selection methods that are analyzed and compared. Through research and analysis, it can be found that the results of ridge regression are better when the model is approximately sparse, SCAD method can better remove the unimportant variables, and the results are very little different from ridge regression. Therefore, this good property can be fully utilized in practical application.