基于非参数方法的分类模型交叉验证结果比较

Comparison of Cross Validation Results of Classification Model Based on Nonparametric Method

摘要: 本文主要研究了基于非参数方法的分类模型交叉验证结果比较，主要是对实例通过非参数的方法进行模型比较的假设检验，检验两分类模型是否存在显著差异。模型的真实泛化误差是一个较为科学的模型比较标准，对于分类模型而言，模型的真实泛化误差表现为分类模型的误判率，而基于交叉验证得到的结果是模型误判率的一个优良估计，可以通过交叉验证结果对模型进行比较。交叉验证结果是随机变量，存在分布，而对于此随机变量而言，其分布是很难观测的，因此，对于交叉验证结果的比较，本文通过非参数的方法进行模型比较的假设检验，检验两分类模型是否存在显著差异。

Abstract: The true generalization error is a scientific evaluation criterion to model selection. For the classi-fication model, the rate of miscarriage of justice, which is an excellent estimation, is based on cross validation to the true generalization error. So we compare models through the cross validation results. Cross validation results are random variables, which have its distribution. For the random variable, its distribution is very hard to detect. Therefore, based on the comparison of cross validation results, this paper designs a hypothesis testing through the nonparametric method to inspect whether a significant difference exists between two classification models.