基于核密度互信息的无监督特征选择方法

Unsupervised Feature Selection Method Based on Kernel Density Mutual Information

摘要: 特征选择是对高维数据进行分析的基础，是许多模式识别应用程序中的一个关键过程。由于使用大量的特征，学习模型会出现过拟合，导致模型性能下降等，因此对高维数据进行特征筛选变得尤为重要。本文提出基于核密度互信息的无监督特征选择方法，算法引入互信息作为特征之间的相关性度量，可以很好地度量特征间的线性和非线性关系，同时采用核密度估计的方式估计特征的密度函数，可以对互信息更准确地计算。最后将该方法应用在两个真实的高维数据集上，并将筛选后的数据应用于三个常用的分类模型，采用分类精度和F1值对分类模型进行评价。通过与现有的基于互相关性方法作对比分析得知，在两个数据集上，两种筛选方法在三个分类模型中的分类精度表现类似，而基于核密度互信息的方法在三种分类算法上的F1值比基于互相关性的方法表现更好，表明本文所提方法具有更好的筛选性能。

Abstract: Feature selection is fundamental to the analysis of high-dimensional data and is a key process in many pattern recognition applications. Due to the use of a large number of features, the learning model will overfit, resulting in the degradation of model performance, etc., so the feature screening of high-dimensional data becomes particularly important. This paper proposes an unsupervised feature selection method based on kernel density mutual information. Mutual information is introduced as the correlation measure between features, which can well measure the linear and nonlinear relationship between features. Finally, the method is applied to two real high-dimensional data sets, and the filtered data is applied to three commonly used classification models, and the classification model is evaluated by classification accuracy and F1 value. Through comparative analysis with the existing cross-correlation-based method, it is known that on the two data sets, the classification accuracy performance of the two screening methods in the three classification models is similar, and the F1 value of the method based on kernel density mutual information is better than the method based on cross-correlation on the three classification algorithms, indicating that the proposed method has better screening performance.