基于最大化联合互信息和最小化联合熵的特征选择

Feature Selection Based on Maximizing Joint Mutual Information and Minimizing Joint Entropy

摘要: 随着大数据时代的到来，数据越来越容易获得，同时获得的数据的维度也越来越高。高维的数据能够详细记录事物的属性，但维度越高，冗余数据也就越多。从数据中剔除冗余特征就显得很重要。基于互信息(MI)的特征选择方法能够有效地降低数据维数和提高分类精度。但现有的方法在特征选择的过程中评判特征的标准单一，无法有效排除冗余特征。本文为此提出了一种基于最大联合互信息和最小联合熵的特征选择方法(JMIMJE)。JMIMJE在特征选择时考虑了整体联合互信息和联合熵两种因素，联合互信息衡量特征子集整体的与分类的相关性，联合熵衡量特征子集的稳定性。JMIMJE在筛选特征时对特征子集的相关性和稳定性进行了平衡。在预测精度方面，JMIMJE比mRMR (最小冗余度最大相关性)提高了2个百分点；与联合互信息(JMI)相比提高了1个百分点。

Abstract: With the advent of the data era, data is getting easier and easier to obtain, and the dimensions of the data are getting higher and higher. Higher-dimensional data can record the attributes of things in detail, but the higher the dimension, the more redundant data. It is important to remove redun-dant features from the data. The feature selection method based on mutual information (MI) is not good at reducing the data dimension and improving the classification accuracy. In the process of feature selection, the existing methods have a single feature evaluation criterion and can not effec-tively eliminate redundant features. A feature selection method (JMIMJE) based on maximum joint mutual information and minimum joint entropy is proposed. JMIMJE considers two factors, global joint mutual information and joint entropy, in feature selection. Combined mutual information to measure the correlation between the whole feature subset and the classification, combined entropy to measure the stability of the feature subset. JMIMJE balances the correlation and stability of fea-ture subsets during feature screening. In terms of prediction accuracy, JMIMJE is 2 percentage points higher than mRMR (minimum redundancy maximum correlation). Compared with Joint Mu-tual Information (JMI), an increase of 1 percentage point.