A Multi-label Image Classification Model Based on Dual Feature Attention

doi:10.3969/j.issn.1006-2475.2013.12.008

Abstract

Abstract: Abstract： A multi-label image classification model based on dual feature attention is proposed to address the current problems of insufficient extraction of feature information from multiple image regions and difficulty in constructing semantic relationships between image features and labels in multi-label image classification tasks. Firstly, the image feature attention module is constructed to correlate the attention of image information with global multi-region features to enhance image feature extraction. Secondly, a combined feature attention module is constructed to perform correlation representation of image feature information and label embedding, thus enabling cross-modal fusion between labels and image regions to obtain a better mapping relationship. The experimental results show that the model achieves better classification results in both the VOC2007 and COCO2014 multi-label image classification datasets, and its performance metrics have improved significantly compared with existing algorithms, verifying the effectiveness of the model.

Key words: Key words： image classification, multi-label, attention mechanisms, deep learning, feature association

CLC Number:

TP391

QIU Kai-xing, FENG Guang. A Multi-label Image Classification Model Based on Dual Feature Attention[J]. Computer and Modernization, 2023, 0(12): 41-47.

References ［29］

［1］	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks［C］// Proceedings of the 2012 International Conference on Neural Information Processing Systems. 2012：1097-1105.
［2］	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv：1409.1556, 2014.
［3］	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2015：1-9.
［4］	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2016：770-778.
［5］	DURAND T, MEHRASA N, MORI G. Learning a deep convNet for multi-label classification with partial labels［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：647-657.
［6］	WEI Y C, XIA W, LIN M, et al. HCP： A flexible CNN framework for multi-label image classification［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38（9）：1901-1907.
［7］	WANG J, YANG Y, MAO J H, et al. CNN-RNN： A unified framework for multi-label image classification［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2016：2285-2294.
［8］	ZHANG J J, WU Q, SHEN C H, et al. Multilabel image classification with regional latent semantic dependencies［J］. IEEE Transactions on Multimedia, 2018,20（10）：2801-2813.
［9］	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks［J］. arXiv preprint arXiv：1609.02907, 2016.
［10］	CHEN Z M, WEI X S, WANG P, et al. Multi-label image recognition with graph convolutional networks［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：5172-5181.
［11］	WANG Y T, XIE Y Z, LIU Y, et al. Fast graph convolution network based multi-label image recognition via cross-modal fusion［C］// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020：1575-1584.
［12］	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017：6000-6010.
［13］	WANG F, JIANG M Q, QIAN C, et al. Residual attention network for image classification［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：6450-6458.
［14］	YAN Z, LIU W W, WEN S P, et al. Multi-label image classification by feature attention network［J］. IEEE Access, 2019,7：98005-98013.
［15］	GUO H, ZHENG K, FAN X C, et al. Visual attention consistency under image transforms for multi-label image classification［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：729-739.
［16］	LANCHANTIN J, WANG T L, ORDONEZ V, et al. General multi-label image classification with transformers［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2021：16473-16483.
［17］	LIU S L, ZHANG L, YANG X, et al. Query2label： A simple transformer way to multi-label classification［J］. arXiv preprint arXiv：2107.10834, 2021.
［18］	LI J L, LI P P, HU X G, et al. Learning common and label-specific features for multi-label classification with correlation information［J］. Pattern Recognition, 2022,121. DOI： 10.1016/j.patcog.2021.108259.
［19］	RIDNIK T, BEN-BARUCH E, ZAMIR N, et al. Asymmetric loss for multi-label classification［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021：82-91.
［20］	EVERINGHAM M, VAN GOOL L, WILLIAMS C K I, et al. The PASCAL visual object classes （VOC） challenge［J］. International Journal of Computer Vision, 2010,88（2）：303-308.
［21］	LIN T Y, Maire M, BELONGIE S, et al. Microsoft COCO： Common objects in context［C］// Proceedings of the 2014 European Conference on Computer Vision （ECCV）. 2014：740-755.
［22］	CUBUK E D, ZOPH B, SHLENS J, et al. Randaugment： Practical automated data augmentation with a reduced search space［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 2020：3008-3017.
［23］	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization［J］. arXiv preprint arXiv：1711.05101, 2017.
［24］	CHEN T S, XU M X, HUI X L, et al. Learning semantic-specific graph representation for multi-label image recognition［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. 2019：522-531.
［25］	HASSANIN M, RADWAN I, KHAN S, et al. Learning discriminative representations for multi-label image recognition［J］. Journal of Visual Communication and Image Representation, 2022,83. DOI： 10.1016/j.jvcir.2022.103448.
［26］	CHEN Z M, WEI X S, WANG P, et al. Learning graph convolutional networks for multi-label recognition and applications［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023,45（6）：6969-6983 .
［27］	ZHU F, LI H S, OUYANG W L, et al. Learning spatial regularization with image-level supervisions for multi-label image classification［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：2027-2036.
［28］	GAO B B, ZHOU H Y. Learning to discover multi-class attentional regions for multi-label image recognition［J］. IEEE Transactions on Image Processing, 2021,30：5920-5932.
［29］	ZHAO J W, YAN K, ZHAO Y F, et al. Transformer-based dual relation graph for multi-label image recognition［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021：163-172.

[1]	HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation [J]. Computer and Modernization, 2024, 0(01): 74-79.
[2]	LIN Wei. Incremental News Recommendation Method Based on Self-supervised Learning and Data Replay [J]. Computer and Modernization, 2023, 0(12): 1-6.
[3]	LIANG Tian-kai, HUANG Kang-hua, LIU Kai-hang, LAN Lan, ZENG Bi. Deep Federated Image Classification Method Based on Bilateral Homomorphic Encryption [J]. Computer and Modernization, 2023, 0(12): 36-40.
[4]	ZHANG Bo-quan, MAI Hai-peng, CHEN Jia-min, Pang Jin-ju. White Matter Hyperintensities Segmentation Based on High Gray Value#br# Attention Mechanism [J]. Computer and Modernization, 2023, 0(12): 67-75.
[5]	WANG Yu-hang, DONG Bao-liang, GONG Chao, SHANG Zhen-zhen, YAO Kang-ning. Dynamic Threat Assessment of Air Swarm Targets Based on Intent Recognition [J]. Computer and Modernization, 2023, 0(12): 100-104.
[6]	TANG Shi-qi, ZHOU Rui-ping, XIE Shi-bin, LIU Meng-chi, XIAO Wen, . Cross-language Multi-label Sentiment Classification Based on Stacked Denoising AutoEncoder [J]. Computer and Modernization, 2023, 0(11): 6-12.
[7]	LI Yan-man, WANG Bi-heng, ZHAO Ling-yan. Safety Helmet Detection Based on Lightweight YOLOv5 [J]. Computer and Modernization, 2023, 0(10): 59-64.
[8]	LI Shi-da, XIANG Jian-wen. A Weakened Joint Reinforcement Method to Improve Robustness of Image Recognition Models [J]. Computer and Modernization, 2023, 0(10): 70-76.
[9]	SHEN Jia-wei, LU Yi-ming, CHEN Xiao-yi, QIAN Mei-ling, LU Wei-zhong, . Review of Research on Human Behavior Detection Methods Based on Deep Learning [J]. Computer and Modernization, 2023, 0(09): 1-9.
[10]	LIU Chan-yi, HUANG Dan, XUE Lin-yan, WANG Tao, ZHU Tao, . COVID-19 X-ray Classification Based on Improved Efficientnet Network [J]. Computer and Modernization, 2023, 0(09): 94-99.
[11]	MA Guo-xiang, YANG Ling-fei, YAN Chuan-bo, ZHANG Zhi-hao, SUN Bing, WANG Xiao-rong. Ultrasonic Image Diagnosis of Hepatic Echinococcosis Based on Deep DenseNet Network [J]. Computer and Modernization, 2023, 0(09): 100-104.
[12]	NONG Hao-cheng, REN De-jun, REN Qiu-lin, LIU Peng-li, HUANG De-cheng. Surface Anomaly Detection Algorithm of Flexible Plastic Packaging Based on Improved ConvNeXt [J]. Computer and Modernization, 2023, 0(08): 12-17.
[13]	OUYANG Fei, WU Xu, XIANG Dong-sheng. Garbage Classification and Detection Method Based on Improved YOLOX [J]. Computer and Modernization, 2023, 0(08): 68-73.
[14]	HU Rui-jie, CHE Dou. Review of Infrared Small Target Detection [J]. Computer and Modernization, 2023, 0(08): 79-86.
[15]	JIANG Lei, TANG Jian, YANG Chao-yue, LYU Ting-ting. Bearing Fault Diagnosis Based on CWGAN-GP and CNN [J]. Computer and Modernization, 2023, 0(07): 1-6.