Feature Weighted Support Vector Machine Based on HSIC Lasso

doi:10.3969/j.issn.1006-2475.2025.07.017

Abstract

Abstract: Abstract: Support vector machine （SVM） has been successfully applied in data classification by transforming the original low-dimensional problem into a high-dimensional linear problem through kernel functions. However， the classical SVM algorithm treats all features equally， ignoring the fact that different features contribute differently to the output of the model. Therefore， the construction of the kernel space may not be entirely reasonable. This paper introduces a feature-weighted SVM algorithm based on the Hilbert-Schmidt independence criterion （HSIC） Lasso， named HSIC Lasso-FWSVM. The algorithm effectively measures the relationship between two random variables using the HSIC， computes the correlation between features， and that between features and labels within the feature space. These correlations are utilized as weights for the corresponding features. Next， the algorithm applies Lasso regression with sparse constraints to reevaluate the weights of various features， shrinking the weights of irrelevant features to zero. Finally， the obtained feature weights are applied to the SVM kernel function calculation， thereby avoiding interference from weakly or non-correlated features during kernel function computation. Experiments were conducted using the proposed algorithm on nine UCI datasets and compared with classical SVM and some recent feature weighted SVM algorithms. The results demonstrate that HSIC Lasso-FWSVM exhibits superior generalization and robustness.

Key words: Key words: support vector machine （SVM）, HSIC Lasso, feature weighting, kernel methods, machine learning

CLC Number:

中图分类号：TP181

LAI Zhiyong, WANG Tinghua, ZHANG Xin. Feature Weighted Support Vector Machine Based on HSIC Lasso[J]. Computer and Modernization, 2025, 0(07): 119-126.

References

［1］张学工. 关于统计学习理论与支持向量机［J］. 自动化学报， 2000，26（1）:32-42.
［2］ BURGES C J C. A tutorial on support vector machines for pattern recognition［J］. Data Mining and Knowledge Discovery， 1998，2（2）:121-167.
［3］ VAPNIK N V. The Nature of Statistical Learning Theory［M］. Springer Science & Business Media， 1999.
［4］ CHAABANE S B， HIJJI M， HARRABI R， et al. Face recognition based on statistical features and SVM classifier［J］. Multimedia Tools and Applications， 2022，81（6）:8767-8784.
［5］ CABA J， BARBA J， RINCON F， et al. Hyperspectral face recognition with adaptive and parallel SVMs in partially hidden face scenarios［J］. Sensors， 2022，22（19）:7641.
［6］ LIU X， WANG S， LU S Y， et al. Adapting feature selection algorithms for the classification of Chinese texts［J］. Systems， 2023，11（9）:483.
［7］ HAO S L， ZHANG P， LIU S， et al. Sentiment recognition and analysis method of official document text based on BERT-SVM model［J］. Neural Computing and Applications， 2023，35:24621-24632.
［8］ RAHMAN M A， HASAN S T， KADER M A. Computer vision based industrial and forest fire detection using support vector machine［C］// Proceedings of the 2022 International Conference on Innovations in Science， Engineering and Technology. IEEE， 2022:233-238.
［9］ SINGH A， BANSAL A， CHAUHAN N， et al. Image generation using GAN and its classification using SVM and CNN［C］// Proceedings of Emerging Trends and Technologies on Intelligent Systems. Springer， 2022:89-100.
［10］ ZHANG Z H， WANG S H， ZHU Z W， et al. Identification of potential feature genes in non-alcoholic fatty liver disease using bioinformatics analysis and machine learning strategies［J］. Computers in Biology and Medicine， 2023，157:106724.
［11］ KAUR A， CHIRE A， WANJALE K， et al. Recognition of protein network for bioinformatics knowledge analysis using support vector machine［J］. BioMed Research International， 2022，2022:2273648.
［12］ NINO-ADAN I， MAANJARRES D， LANDA-TORRES I， et al. Feature weighting methods: A review［J］. Expert Systems with Applications， 2021，184:115424.
［13］ WANG X Z， HE Q. Enhancing generalization capability of SVM classifiers with feature weight adjustment［C］// Proceedings of the 8th International Conference on Knowledge-
based Intelligent Information and Engineering Systems. Springer， 2004:1037-1043.
［14］汪廷华，田盛丰，黄厚宽. 特征加权支持向量机［J］. 电子与信息学报， 2009，31（3）:514-518.
［15］ WANG F， XU Z F， ZHANG W W， et al. Motor imagery classification using geodesic filtering common spatial pattern and filter-bank feature weighted support vector machine［J］. Review of Scientific Instruments， 2020，91（3）:034106.
［16］ HUANG C X， ZHOU J S， CHEN J L， et al. A feature weighted support vector machine and artificial neural network algorithm for academic course performance prediction［J］. Neural Computing and Applications， 2023，35:11517-11529.
［17］ WORDEN K， TSIALIAMANIS G， CROSS E J， et al. Artificial neural networks［M］// Machine Learning in Modeling and Simulation: Methods and Applications. Cham: Springer International Publishing， 2023:85-119.
［18］戴小路，汪廷华，周慧颖. 基于加权马氏距离的模糊多核支持向量机［J］.计算机科学， 2022，49（增2）:302-306
［19］ ALIMI O A， OUAHADA K， ABU-MAHFOUZ A M， et al. Power system events classification using genetic algorithm-based feature weighting technique for support vector machine［J］. Heliyon， 2021，7（1）: e05936.
［20］ MANICKAM M， SIVA R， PRABAKERAN S， et al. Pulmonary disease diagnosis using African vulture optimized weighted support vector machine approach［J］. International Journal of Imaging Systems and Technology， 2022，32（3）:843-856.
［21］胡振威，汪廷华，周慧颖. 基于核统计独立性准则的特征选择研究综述［J］. 计算机工程与应用， 2022，58（22）: 54-64.
［22］ WANG T H， DAI X L， LIU Y Z. Learning with Hilbert-Schmidt independence criterion: A review and new perspectives［J］. Knowledge-Based Systems， 2021，234:107567.
［23］ WANG T H， HU Z W， LIU H M. A unified view of feature selection based on Hilbert-Schmidt independence criterion［J］. Chemometrics and Intelligent Laboratory Systems， 2023，236:104807.
［24］ GRETTON A， FUKUMIZU K， TEO C， et al. A kernel statistical test of independence［C］// Proceedings of the 21st International Conference on Neural Information Processing Systems. NIPS， 2007:585-592.
［25］ GRETTON A， BOUSQUET O， SMOLA A， et al. Measuring statistical dependence with Hilbert-Schmidt norms ［C］// Proceedings of the 16th International Conference on Algorithmic Learning Theory. Springer， 2005:63-77.
［26］ YAMADA M， JITKRITTUM W， SIGAL L， et al. High-dimensional feature selection by feature-wise kernelized Lasso［J］. Neural Computation， 2014，26（1）:185-207.
［27］ WANG X Z， WANG Y D， WANG L J. Improving fuzzy c-means clustering based on feature-weight learning［J］. Pattern Recognition Letters， 2004，25（10）:1123-1132.
［28］ KELLY， M， LONGJOHN R， NOTTINGHAM K. The UCI Machine Learning Repository ［DB/OL］. ［2024-03-10］. https://archive.ics.uci.edu.
［29］ PEDREGOSA F， VAROQUAUX G， GRAMFORT A， et al. Scikit-learn: Machine learning in Python［J］. Journal of Machine Learning Research， 2011，12:2825-2830.

[1]	DENG Chao1, 2, YANG Fengkun1, 2, LI Jun3, CHEN Liangliang1, 2, GUO Zhichong4. Complex Fault Diagnosis Method of Charging Facilities Based on Bi-LSTM Residual Network [J]. Computer and Modernization, 2025, 0(07): 90-96.
[2]	XU Shengchao, CHEN Fuqiang. A Food Safety Risk Warning Method Based on BP Neural Network [J]. Computer and Modernization, 2025, 0(01): 20-24.
[3]	JIA Xiao-yao, . Breast Cancer Prediction and Feature Analysis Model Based on CatBoost and SHAP [J]. Computer and Modernization, 2023, 0(10): 32-38.