S2R2: Semi-supervised Feature Selection Based on Analysis of Relevance and Redundancy

Abstract

Abstract: Feature selection is one of the key problems of pattern recognition and data mining, which can be removed dataset redundant and irrelevant features to improve learning performance. Based on the max-relevance and min-redundancy criteria, a novel semi-supervised feature selection method based on relevance and redundancy analysis is proposed. This new method is independent of any classification learning algorithm. Firstly, unsupervised relevance is analyzed and expanded. Then it is combined with information gain to form a semi-supervised feature relevance and redundancy measures, which can effectively identify and remove irrelevant and redundant features. Finally, an incremental forward search is used to construct feature subset in a greedy manner, which avoiding the search for exponential solution spaces and improving algorithm efficiency. This article also proposes the FS2R2 method as a fast version of the S2R2 method to deal with large-scale problems. The experimental results on standard data sets illustrate the effectiveness and superiority of the proposed approaches.

Key words: semi-supervised learning, feature selection, information theory, max-relevance and min-redundancy

ZHANG Dong-fang, CHEN Hai-yan, YUAN Li-gang. S2R2: Semi-supervised Feature Selection Based on Analysis of Relevance and Redundancy[J]. Computer and Modernization, 2021, 0(09): 113-120.

References

［1］ SREEDHARAN N P, GANESAN B, RAVEENDRAN R, et al. Grey wolf optimisation-based feature selection and classification for facial emotion recognition［J］. IET Biometrics, 2018,7(5):490-499.
［2］吴成东,卢紫微,于晓升. 基于加权随机森林的图像超分辨率算法研究［J］. 控制与决策, 2019,34(10):2243-2248.
［3］ FANG S Q, CAI Z P, SUN W C, et al. Feature selection method based on class discriminative degree for intelligent medical diagnosis［J］. Computers Materials & Continua, 2018,55(3):419-433.
［4］ SOMU N, RAMAN M R, KIRTHIVASAN K, et al. Hypergraph based feature selection technique for medical diagnosis［J］. Journal of Medical Systems, 2016,40(11):1-16.〖HJ1.65mm〗
［5］ TUNC A. Feature selection in credibility study for finance sector［J］. Procedia Computer Science, 2019,158:254-259.
［6］ ZENG Z L, ZHANG H J, ZHANG R, et al. A novel feature selection method considering feature interaction［J］. Pattern Recognition, 2015,48(8):2656-2666.
［7］ MWANGI B, TIAN T S, SOARES J C, et al. A review of feature reduction techniques in neuroimaging［J］. Neuroinformatics, 2014,12(2):229-244.
［8］ SUGIYAMA M. Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis［J］. Journal of Machine Learning Research, 2007(8):1027-1061.
［9］ WANG Y T, WANG J D, CHEN H Y, et al. Semi-supervised local fisher discriminant analysis based on reconstruction probability class［J］. International Journal of Pattern Recognition and Artificial Intelligence, 2015,29(2):1550007.
［10］PENG H C, LONG F H, DING C, et al. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005,27(8):1226-1238.
［11］CHANDRASHEKAR G, SAHIN F. A survey on feature selection methods［J］. Computers & Electrical Engineering, 2014,40(1):16-28.
［12］XUE B, ZHANG M J, BROWNE W N, et al. A survey on evolutionary computation approaches to feature selection［J］. IEEE Transactions on Evolutionary Computation, 2016,20(4):606-626.
［13］毛勇,周晓波,夏铮,等. 特征选择算法研究综述［J］. 模式识别与人工智能, 2007,20(2):211-218.
［14］CAI J, LUO J W, WANG S l, et al. Feature selection in machine learning: A new perspective［J］. Neurocomputing, 2018,300:70-79.
［15］YU L, LIU H. Efficient feature selection via analysis of relevance and redundancy［J］. Journal of Machine Learning Research, 2004,5:1205-1224.
［16］XU J L. Adapt the mRMR criterion for unsupervised feature selection［C］// International Conference on Advanced Data Mining and Applications, 2010:111-121.
［17］XU J, TANG B, HE H B, et al. Semisupervised feature selection based on relevance and redundancy criteria［J］. IEEE Transactions on Neural Networks, 2017,28(9):1974-1984.
［18］ZHAO J D, LU K, HE X F, et al. Locality sensitive semi-supervised feature selection［J］. Neurocomputing, 2008,71(10):1842-1849.

［19］LEE S, PARK Y, DAURIOL B J, et al. A novel feature selection method based on normalized mutual information［J］. Applied Intelligence, 2012,37(1):100-120.

［20］GAO W F, HU L, ZHANG P, et al. Class-specific mutual information variation for feature selection［J］. Pattern Recognition, 2018,79:328-339.
［21］WANG Y T, WANG J D, LIAO H, et al. An efficient semi-supervised representatives feature selection algorithm based on information theory［J］. Pattern Recognition, 2017,61:511-523.
［22］KOLLER D, SAHAMI M. Toward optimal feature selection［C］// International Conference on Machine Learning. 1996:284-292.
［23］PRESS H, TEUKOLSKY A, VETTERLINGT, et al. Numerical Recipes in C［M］. Cambridge: Cambridge University Press, 1996.
［24］ASUNCION A. UCI Machine Learning Repository［DB/OL］. ［2021-01-08］. http://archive.ics.uci.edu/ml/datasets.html.
［25］ZHAO Z, MORSTATTER F, SHARMA S, et al. Advancing feature selection research［J］. ASU Feature Selection Repository Arizona State University, 2010:1-28.
［26］FAYYAD U M, IRANIK B. Multi-interval discretization of continuous-valued attributes for classification learning［C］// International Joint Conference on Artificial Intelligence. 1993:1022-1027.
［27］FIX E, HODGES J L. Discriminatory analysis nonparametric discrimination: Consistency properties［J］. International Statistical Review, 1989,57(3):21-49.
［28］ZHANG S C, LI X L, ZONG M, et al. Efficient kNN classification with different numbers of nearest neighbors［J］. IEEE Transactions on Neural Networks and Learning Systems, 2018,29(5):1774-1785.
［29］SALZBERG S L. C4.5: Programs for Machine Learning［M］. Elsevier, 2014.
［30］SAXENA A, PARE S, MEENA M S, et al. A two-phase approach for semi-supervised feature selection［J］. Algorithms, 2020,13(9):215.

[1]	SU Kaixuan . Network Intrusion Detection Based on Improved XGBoost Model [J]. Computer and Modernization, 2024, 0(06): 109-114.
[2]	LUO Shuhuan, SUN Wu, YOU Jie, WANG Wei, HU Biwei, JIANG Nan. Non-destructive Detection of Total Acid Content in Pear Based on#br# Visible-near Infrared Spectroscopy [J]. Computer and Modernization, 2024, 0(05): 80-84.
[3]	LIU Ying-jie, LAN Hai, WEI Xian. Semi-supervised Learning Method Based on Convolution and Sparse Coding [J]. Computer and Modernization, 2022, 0(11): 9-16.
[4]	WANG Yang, CHEN Mei, LI Hui. FOCoR: A Course Recommendation Approach Based on Feature Selection Optimization [J]. Computer and Modernization, 2022, 0(10): 1-7.
[5]	ZHAO Ying-li, ZHU Xu. Association Analysis of Image Genetic Data Based on Group Sparse Joint Leraning [J]. Computer and Modernization, 2022, 0(08): 43-49.
[6]	MO Yun. EEG Decoding Method Based on Hybrid Feature Selection [J]. Computer and Modernization, 2022, 0(04): 92-96.
[7]	ZHAO Yan-ping, WANG Fang, XIA Yang. Short Text Classification Method Based on Support Vector Machine [J]. Computer and Modernization, 2022, 0(02): 92-96.
[8]	DAI Ji-peng, SHAO Feng-jing, SUN Ren-cheng. Short Text Classification Based on Improved CHI and TF-IDF [J]. Computer and Modernization, 2021, 0(06): 6-11.
[9]	CHEN Si-yu, ZHUANG Yi, LI Jing. Multi Feature Load Forecasting Model for LSTM Network in Mobile Cloud Computing [J]. Computer and Modernization, 2021, 0(06): 74-85.
[10]	YANG Qiu-liang, WANG Yu, YANG Xing-li, LI Ji-hong. Classification of Ground-based Meteorological Cloud Images Based on Feature Selection Technique of Mutual Information F-statistics [J]. Computer and Modernization, 2021, 0(02): 18-23.
[11]	JIANG Wan-ming1,2, GUO Chun1,2, JIANG Chao-hui1,2. A Low-rate DDoS Attack Detection Method Based on BiLSTM [J]. Computer and Modernization, 2020, 0(05): 120-.
[12]	MA Ji-ke, YIN Fei, ZHU Yong-jin, DOU Long-long, LI Jian. A Semi-supervised Learning Method for Operating State Identification of #br# Energy Measurement Devices [J]. Computer and Modernization, 2020, 0(03): 82-.
[13]	YAN Yi, LI Bo, CHEN Shou-ming, LIN Qiang, HUANG Ju-tao, Wen Bo-jian. A Causal Feature Selection Algorithm for Feedback Networks and Its Applications [J]. Computer and Modernization, 2019, 0(12): 95-.
[14]	LI Fu-xing，MENG Zu-qiang . An Improved Feature Selection Algorithm Based on Category Distinguished Words [J]. Computer and Modernization, 2019, 0(03): 73-.
[15]	MA Jian-hong， LIU Guang-sen， YAO Shuang， YANG Zhi . Text Feature Selection and Text Representation for Short Essays [J]. Computer and Modernization, 2019, 0(03): 95-.