基于HSIC Lasso的特征加权支持向量机

doi:10.3969/j.issn.1006-2475.2025.07.017

计算机与现代化 ›› 2025, Vol. 0 ›› Issue (07): 119-126.doi: 10.3969/j.issn.1006-2475.2025.07.017

• 算法设计与分析 • 上一篇

基于HSIC Lasso的特征加权支持向量机

（赣南师范大学数学与计算机科学学院，江西赣州 341000）

出版日期:2025-07-22 发布日期:2025-07-22
作者简介: 作者简介：赖志勇（1997—），男，江西赣州人，硕士研究生，研究方向:机器学习与数据挖掘，E-mail: jonezylai@163.com；通信作者:汪廷华（1977—），男，江西玉山人，教授，博士，研究方向:人工智能与机器学习，E-mail: wthpku@163.com；张昕（1999—），男，江西赣州人，硕士研究生，研究方向:机器学习与数据挖掘，E-mail: zhxblbj@163.com。
基金资助:
基金项目：江西省自然科学基金重点项目（20242BAB26024）；江西省研究生创新专项资金资助项目（YC2023-S865）；江西省学位与研究生教育教学改革研究项目（JXYJG-2022-172）

Feature Weighted Support Vector Machine Based on HSIC Lasso

（School of Mathematics and Computer Science， Gannan Normal University， Ganzhou 341000， China）

Online:2025-07-22 Published:2025-07-22

摘要/Abstract

摘要： 摘要：支持向量机（SVM）通过引入核函数将原低维问题转化为高维核空间线性问题，已成功应用于数据分类等问题。经典的SVM算法平等对待所有的特征，忽略了不同的特征对模型输出的贡献不一样的事实，因此，核空间的构造可能不尽合理。本文提出一种基于希尔伯特—施密特独立性准则（Hilbert-Schmidt Independence Criterion， HSIC）Lasso的特征加权支持向量机算法HSIC Lasso-FWSVM。该算法首先利用HSIC有效地度量2个随机变量之间的关系，计算特征空间中特征与特征以及特征与标签之间的相关性，使用计算出的相关性作为对应特征的权重。然后应用Lasso回归方法，通过稀疏性约束重新度量各个特征的权重，将部分无关特征的权重收缩至0。最后，将得到的特征权重应用到SVM核函数的计算中，从而避免弱相关或不相关的特征干扰核函数的计算。在9个UCI数据集上分别使用本文算法、经典SVM和一些最新的特征加权SVM算法进行仿真实验，结果表明，HSIC Lasso-FWSVM具有更好的泛化能力和鲁棒性。

关键词: 关键词：支持向量机, HSIC Lasso, 特征加权, 核方法, 机器学习

Abstract: Abstract: Support vector machine （SVM） has been successfully applied in data classification by transforming the original low-dimensional problem into a high-dimensional linear problem through kernel functions. However， the classical SVM algorithm treats all features equally， ignoring the fact that different features contribute differently to the output of the model. Therefore， the construction of the kernel space may not be entirely reasonable. This paper introduces a feature-weighted SVM algorithm based on the Hilbert-Schmidt independence criterion （HSIC） Lasso， named HSIC Lasso-FWSVM. The algorithm effectively measures the relationship between two random variables using the HSIC， computes the correlation between features， and that between features and labels within the feature space. These correlations are utilized as weights for the corresponding features. Next， the algorithm applies Lasso regression with sparse constraints to reevaluate the weights of various features， shrinking the weights of irrelevant features to zero. Finally， the obtained feature weights are applied to the SVM kernel function calculation， thereby avoiding interference from weakly or non-correlated features during kernel function computation. Experiments were conducted using the proposed algorithm on nine UCI datasets and compared with classical SVM and some recent feature weighted SVM algorithms. The results demonstrate that HSIC Lasso-FWSVM exhibits superior generalization and robustness.

Key words: Key words: support vector machine （SVM）, HSIC Lasso, feature weighting, kernel methods, machine learning

中图分类号:

中图分类号：TP181

赖志勇, 汪廷华, 张昕. 基于HSIC Lasso的特征加权支持向量机[J]. 计算机与现代化, 2025, 0(07): 119-126.

LAI Zhiyong, WANG Tinghua, ZHANG Xin. Feature Weighted Support Vector Machine Based on HSIC Lasso[J]. Computer and Modernization, 2025, 0(07): 119-126.

参考文献

［1］张学工. 关于统计学习理论与支持向量机［J］. 自动化学报， 2000，26（1）:32-42.
［2］ BURGES C J C. A tutorial on support vector machines for pattern recognition［J］. Data Mining and Knowledge Discovery， 1998，2（2）:121-167.
［3］ VAPNIK N V. The Nature of Statistical Learning Theory［M］. Springer Science & Business Media， 1999.
［4］ CHAABANE S B， HIJJI M， HARRABI R， et al. Face recognition based on statistical features and SVM classifier［J］. Multimedia Tools and Applications， 2022，81（6）:8767-8784.
［5］ CABA J， BARBA J， RINCON F， et al. Hyperspectral face recognition with adaptive and parallel SVMs in partially hidden face scenarios［J］. Sensors， 2022，22（19）:7641.
［6］ LIU X， WANG S， LU S Y， et al. Adapting feature selection algorithms for the classification of Chinese texts［J］. Systems， 2023，11（9）:483.
［7］ HAO S L， ZHANG P， LIU S， et al. Sentiment recognition and analysis method of official document text based on BERT-SVM model［J］. Neural Computing and Applications， 2023，35:24621-24632.
［8］ RAHMAN M A， HASAN S T， KADER M A. Computer vision based industrial and forest fire detection using support vector machine［C］// Proceedings of the 2022 International Conference on Innovations in Science， Engineering and Technology. IEEE， 2022:233-238.
［9］ SINGH A， BANSAL A， CHAUHAN N， et al. Image generation using GAN and its classification using SVM and CNN［C］// Proceedings of Emerging Trends and Technologies on Intelligent Systems. Springer， 2022:89-100.
［10］ ZHANG Z H， WANG S H， ZHU Z W， et al. Identification of potential feature genes in non-alcoholic fatty liver disease using bioinformatics analysis and machine learning strategies［J］. Computers in Biology and Medicine， 2023，157:106724.
［11］ KAUR A， CHIRE A， WANJALE K， et al. Recognition of protein network for bioinformatics knowledge analysis using support vector machine［J］. BioMed Research International， 2022，2022:2273648.
［12］ NINO-ADAN I， MAANJARRES D， LANDA-TORRES I， et al. Feature weighting methods: A review［J］. Expert Systems with Applications， 2021，184:115424.
［13］ WANG X Z， HE Q. Enhancing generalization capability of SVM classifiers with feature weight adjustment［C］// Proceedings of the 8th International Conference on Knowledge-
based Intelligent Information and Engineering Systems. Springer， 2004:1037-1043.
［14］汪廷华，田盛丰，黄厚宽. 特征加权支持向量机［J］. 电子与信息学报， 2009，31（3）:514-518.
［15］ WANG F， XU Z F， ZHANG W W， et al. Motor imagery classification using geodesic filtering common spatial pattern and filter-bank feature weighted support vector machine［J］. Review of Scientific Instruments， 2020，91（3）:034106.
［16］ HUANG C X， ZHOU J S， CHEN J L， et al. A feature weighted support vector machine and artificial neural network algorithm for academic course performance prediction［J］. Neural Computing and Applications， 2023，35:11517-11529.
［17］ WORDEN K， TSIALIAMANIS G， CROSS E J， et al. Artificial neural networks［M］// Machine Learning in Modeling and Simulation: Methods and Applications. Cham: Springer International Publishing， 2023:85-119.
［18］戴小路，汪廷华，周慧颖. 基于加权马氏距离的模糊多核支持向量机［J］.计算机科学， 2022，49（增2）:302-306
［19］ ALIMI O A， OUAHADA K， ABU-MAHFOUZ A M， et al. Power system events classification using genetic algorithm-based feature weighting technique for support vector machine［J］. Heliyon， 2021，7（1）: e05936.
［20］ MANICKAM M， SIVA R， PRABAKERAN S， et al. Pulmonary disease diagnosis using African vulture optimized weighted support vector machine approach［J］. International Journal of Imaging Systems and Technology， 2022，32（3）:843-856.
［21］胡振威，汪廷华，周慧颖. 基于核统计独立性准则的特征选择研究综述［J］. 计算机工程与应用， 2022，58（22）: 54-64.
［22］ WANG T H， DAI X L， LIU Y Z. Learning with Hilbert-Schmidt independence criterion: A review and new perspectives［J］. Knowledge-Based Systems， 2021，234:107567.
［23］ WANG T H， HU Z W， LIU H M. A unified view of feature selection based on Hilbert-Schmidt independence criterion［J］. Chemometrics and Intelligent Laboratory Systems， 2023，236:104807.
［24］ GRETTON A， FUKUMIZU K， TEO C， et al. A kernel statistical test of independence［C］// Proceedings of the 21st International Conference on Neural Information Processing Systems. NIPS， 2007:585-592.
［25］ GRETTON A， BOUSQUET O， SMOLA A， et al. Measuring statistical dependence with Hilbert-Schmidt norms ［C］// Proceedings of the 16th International Conference on Algorithmic Learning Theory. Springer， 2005:63-77.
［26］ YAMADA M， JITKRITTUM W， SIGAL L， et al. High-dimensional feature selection by feature-wise kernelized Lasso［J］. Neural Computation， 2014，26（1）:185-207.
［27］ WANG X Z， WANG Y D， WANG L J. Improving fuzzy c-means clustering based on feature-weight learning［J］. Pattern Recognition Letters， 2004，25（10）:1123-1132.
［28］ KELLY， M， LONGJOHN R， NOTTINGHAM K. The UCI Machine Learning Repository ［DB/OL］. ［2024-03-10］. https://archive.ics.uci.edu.
［29］ PEDREGOSA F， VAROQUAUX G， GRAMFORT A， et al. Scikit-learn: Machine learning in Python［J］. Journal of Machine Learning Research， 2011，12:2825-2830.

[1]	邓超1, 2, 杨凤坤1, 2, 李珺3, 陈良亮1, 2, 郭志冲4. 融合Bi-LSTM残差网络的充电设施复杂故障诊断方法[J]. 计算机与现代化, 2025, 0(07): 90-96.
[2]	牛奉高, 闫涛. 特征加权的CLSVSM[J]. 计算机与现代化, 2021, 0(05): 59-65.
[3]	常雪，石鸿雁. 基于改进蝙蝠算法优化的FCM聚类算法[J]. 计算机与现代化, 2020, 0(05): 29-.

基于HSIC Lasso的特征加权支持向量机

Feature Weighted Support Vector Machine Based on HSIC Lasso

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价