Boundary Mixed Resampling Based on Joint Entropy for Imbalanced Data

doi:10.3969/j.issn.1006-2475.2024.09.016

Abstract

Abstract: In order to overcome the limitations of single resampling methods in data imbalance handling， which often lead to the generation of redundant samples and the inadvertent deletion of crucial sample information， this paper proposes a novel non-balanced data boundary mixed resampling algorithm based on joint entropy. The algorithm first effectively distinguishes between the boundary set and the non-boundary set by introducing a boundary factor. It further constructs a joint entropy indicator system to assess the importance of minority class samples within the boundary set. Based on this assessment， different oversampling methods and sampling quantities are applied to the segmented minority class samples. Finally， the NearMiss-2 algorithm is used to filter and remove most of the sample points in the non-boundary set， thus achieving a relative data balance. Through comparative experiments on nine sets of UCI datasets， the experimental results show that the proposed algorithm achieves improvements in F1-Score， G-mean， and AUC metrics， which validates its effectiveness and exhibiting favorable performance in non-balanced data classification.

Key words: , imbalanced data classification； boundary factor； joint entropy； mixed sampling

CLC Number:

TP301.6

ZHOU Chuanhua1, 2, REN Taijiao1, LUO Lan1, ZHOU Hao1. Boundary Mixed Resampling Based on Joint Entropy for Imbalanced Data[J]. Computer and Modernization, 2024, 0(09): 95-100.

References

［1］ HE H B， GARCIA E A. Learning from imbalanced data ［J］. IEEE Transactions on Knowledge and Data Engineering， 2009，21（9）：1263-1284.
［2］ ZHAO H， WANG R X， LEI Y G， et al. Severity level diagnosis of Parkinson’s disease by ensemble K-nearest neighbor under imbalanced data［J］. Expert Systems with Applications， 2022，189. DOI： 10.1016/j.eswa.2021.116113.
［3］ PANDEY S， KUMAR K. Software fault prediction for imbalanced data： A survey on recent developments［J］. Procedia Computer Science， 2023，218：1815-1824.
［4］ LI Z C， HUANG M， LIU G J， et al. A hybrid method with dynamic weighted entropy for handling the problem of class imbalance with overlap in credit card fraud detection［J］. Expert Systems with Applications， 2021，175. DOI：10.1016
/j.eswa.2021.114750.
［5］ GOODFELLOW I， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial networks［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. ACM， 2014：2672-2680.
［6］赵海霞，石洪波，武建，等. 基于条件生成对抗网络的不平衡学习研究［J］. 控制与决策， 2021，36（3）：619-628.
［7］ MORENO-BAREA F J， JEREZ J M， FRANCO L. Improving classification accuracy using data augmentation on small data sets［J］. Expert Systems with Applications， 2020，161. DOI： 10.1016/j.eswa.2020.113696.
［8］李艳霞，柴毅，胡友强，等. 不平衡数据分类方法综述［J］. 控制与决策， 2019，34（4）：673-688.
［9］ SUN Y， KAMEL M S， WONG A K， et al. Cost-sensitive boosting for classification of imbalanced data［J］. Pattern Recognition， 2007，40（12）：3358-3378.
［10］ TAO X M， LI Q， GUO W J， et al. Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification［J］. Information Sciences， 2019，487：31-56.
［11］ CHAWLA N V， BOWYER K W， HALL L O， et al. SMOTE： Synthetic minority over-sampling technique［J］. Journal of Artificial Intelligence Research， 2002，16：321-357.
［12］ HAN H， WANG W Y， MAO B H. Borderline-SMOTE： A new over-sampling method in imbalanced data sets learning［C］// Proceedings of the 2005 International Conference on Intelligent Computing. Springer， 2005：878-887.
［13］ BUNKHUMPORNPAT C， SINAPIROMSARAN K， LURSINSAP C. Safe-Level-SMOTE： Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem［C］// Proceedings of the 13th Pacific-Asia Conference on Knowledge Discovery & Data Mining. Springer-Verlag， 2009：475-482.
［14］ PRADIPTA G A， WARDOYO R， MUSDHOLIFAH A， et al. Radius-SMOTE： A new oversampling technique of minority samples based on radius distance for learning from imbalanced data［J］. IEEE Access， 2021，9：74763-74777.
［15］ SOWAH R A， AGEBURE M A， MILLS G A， et al. New cluster undersampling technique for class imbalance learning［J］. International Journal of Machine Learning and Computing， 2016，6（3）：205-214.
［16］ ZHANG J P， MANI I. kNN approach to unbalanced data distributions： A case study involving information extraction［C］// Proceedings of the ICML 2003 Workshop on Learning from Imbalanced Datasets. ICML， 2003，126：1-7.
［17］ GAZZAH S， HECHKEL A， AMARA N E B. A hybrid sampling method for imbalanced data［C］// 2015 IEEE 12th International Multi-Conference on Systems， Signals & Devices （SSD15）. IEEE， 2015. DOI: 10.1109/SSD.2015.7348093.
［18］ BATISTA G E， PRATI R C， MONARD M C. A study of the behavior of several methods for balancing machine learning training data［J］. ACM SIGKDD Explorations Newsletter， 2004，6（1）：20-29.
［19］ GUAN H J， ZHANG Y T， XIAN M， et al. SMOTE-WENN： Solving class imbalance and small sample problems by oversampling and distance scaling［J］. Applied Intelligence， 2021，51：1394-1409.
［20］侯贝贝，刘三阳，普事业. 基于边界混合重采样的非平衡数据分类方法［J］. 计算机工程与应用， 2020，56（1）：46-52.
［21］崔鑫，徐华，宿晨. 面向不均衡数据集的过抽样算法［J］. 计算机应用， 2020，40（6）：1662-1667.
［22］李亦农，李梅. 信息论基础教程［M］. 北京：北京邮电大学出版社， 2005.
［23］邱保志，曹鹤玲. 一种高效的基于联合熵的边界点检测算法［J］. 控制与决策， 2011，26（1）：71-74.
［24］ LIU Y， ZHENG Z Y， ZHAO L， et al. Quality assessment of post-consumer plastic bottles with joint entropy method： A case study in Beijing， China［J］. Resources， Conservation and Recycling， 2021，175. DOI：10.1016/j.resconrec.2021.105839.
［25］姜新盈，江开忠，严涛，等. 不平衡数据中基于权重的边界混合采样［J］. 计算机工程与设计， 2022，43（5）：1265-1272.
［26］周传华，朱俊杰，徐文倩，等. 基于聚类欠采样的集成分类算法［J］. 计算机与现代化， 2021（11）：72-76.
［27］孙丹，施炜利，饶兰香，等. 基于改进混合采样和XGBoost算法的信用卡欺诈检测方法［J］. 计算机与现代化， 2022（9）：111-118.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[3]	WANG Xiaohang1, LI Yongjie1, YU Lei1, FAN Xiao2. A Method of Using Compound Event Probability Operation to Solve Problem of Negative Information Blocking Maximization [J]. Computer and Modernization, 2024, 0(12): 24-33.
[4]	ZHANG Xiaodong1, BAI Guangzhi1, LI Min1, LI Haoyang2. Oil and Gas Well Production Prediction Model Based on Empirical Wavelet Transform [J]. Computer and Modernization, 2024, 0(12): 53-58.
[5]	LIU Yunhai1, Feng Guang1, WU Xiaoting2, YANG Qun2 . Safety Helmet Wearing Detection Algorithm for Complex Construction Scenes [J]. Computer and Modernization, 2024, 0(12): 66-71.
[6]	LIU Baobao, YANG Jingjing, TAO Lu, WANG Heying . DSMSC Based on Attention Mechanism for Remote Sensing Image Scene Classification [J]. Computer and Modernization, 2024, 0(12): 72-77.
[7]	GU Yue, DENG Songfeng, SHEN Ji, MU Wentao, ZHAO Enqi. SAR Ship Detection Algorithm Based on Improved YOLOv8 [J]. Computer and Modernization, 2024, 0(12): 78-83.
[8]	WU Xiuling1, ZHOU Sheng1, WANG Chunjuan1, YU Cuizhuo2, LIU Hao3. Research Progress in Ultra Short-term Power Load Forecasting Technology [J]. Computer and Modernization, 2024, 0(12): 108-115.
[9]	LI Deyou1, 2, YU Jinsongdi1, 2, WEI Dandan1, 2, LUO Yuan1, 2, TONG Ruiju3. Abstract Tree Model for Gridded Cube Metadata [J]. Computer and Modernization, 2024, 0(11): 1-6.
[10]	GONG Yicheng1, 2, LIU Qing1, 2. Beijing Opera Binary Classification Based on RF-LCE-BiLSTM-Attention-AMSSA Model [J]. Computer and Modernization, 2024, 0(11): 7-12.
[11]	LI Taoying, LI Meng, WU Mengqiao. Taxi Passenger Flow Prediction Based on Heterogeneous Spatiotemporal Graph#br# Convolutional Networks [J]. Computer and Modernization, 2024, 0(11): 13-18.
[12]	ZHANG Tai1, YAN Zihao2, DUAN Jie2, ZHANG Zhihong2. Information Forwarding Strategy of Internet of Vehicles in Named Data Network [J]. Computer and Modernization, 2024, 0(11): 19-27.
[13]	YUAN Qingle, MU Li. Inventory Forecasting Method Based on Improved Elman Neural Network [J]. Computer and Modernization, 2024, 0(11): 28-33.
[14]	ZHANG Kun1, ZHANG Yongwei1, WU Yongcheng1, ZHANG Xiaowen2, ZHAI Shichen2. An LLM-based Method for Automatic Construction of Equipment Failure Knowledge Graphs [J]. Computer and Modernization, 2024, 0(11): 46-53.
[15]	YE Xue, YANG Sheng, CHENG Kai, ZHU Feng. A Financial Knowledge Q&A Model for Power Enterprise Based on ChatGLM2-6B [J]. Computer and Modernization, 2024, 0(11): 54-63.