基于自蒸馏和自注意力增强的低数据细粒度图像分类

doi:10.3969/j.issn.1006-2475.2025.09.004

摘要/Abstract

摘要：
摘要：利用有限的数据训练细粒度图像分类（Fine-grained Image Classification， FGIC）模型是一项巨大的挑战，其类别之间的细微差异可能不易辨别。一种常用的策略是利用预训练的网络模型生成有效的特征表示。然而当使用有限的细粒度数据对预训练模型进行微调时，模型往往倾向于提取相关性较低的特征，从而引发过拟合问题。针对上述问题，本文设计一种新的低数据条件下的细粒度图像分类方法SDA-Net。该方法通过融合空间自注意力机制和自蒸馏技术优化特征学习过程，能有效减轻因数据稀缺导致的过拟合问题，提升深度神经网络在低数据环境下的性能表现。具体而言，SDA-Net通过引入空间自注意力将上下文信息编码为局部特征，改进类内表示。同时引入一个蒸馏分支并将蒸馏损失用于增广后的输入样本中，实现网络内部知识的深度增强与传递。在3个细粒度基准数据上进行了全面的评估，结果显示SDA-Net相较于传统的微调方法及当前最先进的低数据FGIC技术，均展现出了显著的性能提升。在3种10%的低数据场景下，与标准ResNet-50相比，相对精度分别提高了30%、47%和29%；与SOTA相比，相对精度分别提高了15%、28%和17%。

关键词: 关键词：深度学习, 细粒度图像分类, 低数据学习, 自蒸馏, 自注意力, 数据增广

Abstract: Abstract: Training a fine-grained image classification （FGIC） model with limited data is a great challenge， where subtle differences between categories may not be easily discernible. A common strategy is to utilize pre-trained network models to generate effective feature representations. However， when fine-tuning the pre-trained model using limited fine-grained data， the model often tends to extract less relevant features， which triggers the overfitting problem. To address the above issues， this paper designs an new FGIC method named SDA-Net under low-data conditions， which optimizes the feature learning process by fusing the spatial self-attention mechanism and the self-distillation technique， which can effectively mitigate the overfitting problem caused by data scarcity and improve the performance of deep neural networks in low-data environments. Specifically， SDA-Net improves the intra-class representation by introducing spatial self-attention to encode contextual information into local features. Meanwhile， a distillation branch is introduced and the distillation loss is used in the augmented input samples， which realizes the deep enhancement and transfer of knowledge within the network. A comprehensive evaluation on three fine-grained benchmark data shows that SDA-Net exhibits significant performance gains compared to both traditional fine-tuning methods and the current SOTA low-data FGIC strategy. In 3 scenarios with 10% low-data volume， relative accuracies are improved by 30%， 47%， and 29%， respectively， compared to standard ResNet-50， and by 15%， 28%， and 17%， respectively， compared to SOTA.

Key words: Key words: deep learning, fine-grained image classification, low-data learning, self-distillation, self-attention, data augmentation

中图分类号:

TP391

张婧颖1, 耿琳2, 刘宁钟2. 基于自蒸馏和自注意力增强的低数据细粒度图像分类[J]. 计算机与现代化, 2025, 0(09): 27-34.

ZHANG Jingying1, GENG Lin2, LIU Ningzhong2. Low-data Fine-grained Image Classification Based on Self-distillation and Self-attention Enhancement[J]. Computer and Modernization, 2025, 0(09): 27-34.

参考文献

［1］ JIA D， WEI D， RICHARD S， et al. ImageNet: A large-scale hierarchical image database［C］// Proceedings of 2009 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2009:248-255.
［2］ SCHMARJE L， SANTAROSSA M， SCHRÖDER S M， et al. A survey on semi-self-and unsupervised learning for image classification［J］. IEEE Access， 2021，9:82146-82168.
［3］ XU M D， ZHANG Z， HU H， et al. End-to-end semi-supervised object detection with soft teacher［C］// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. IEEE， 2021:3060-3069.
［4］ YANG X L， SONG Z X， KING I， et al. A survey on deep semi-supervised learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2022，35（9）:8934-8954.
［5］ SHU Y Y， YU B S， XU H M， et al. Improving fine-grained visual recognition in low data regimes via self-boosting attention mechanism［C］// European Conference on Computer Vision. Springer， 2022:449-465.
［6］马瑶，智敏，殷雁君，等. CNN 和 Transformer 在细粒度图像识别中的应用综述［J］. 计算机工程与应用， 2022，58（19）:53-63
［7］ DEMIDOV D， AL MAJZOUB R， KUMAR A， et al. Distilling local texture features for colorectal tissue classification in low data regimes［C］// International Workshop on Machine Learning in Medical Imaging. Springer， 2023:357-366.
［8］ SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556， 2014.
［9］ YUN S， HAN D， OH S J， et al. CutMix: Regularization strategy to train strong classifiers with localizable features［C］// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. IEEE， 2019:6023-6032.
［10］ HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016:770-778.
［11］申志军，穆丽娜，高静，等. 细粒度图像分类综述［J］. 计算机应用， 2023，43（1）:51-60.
［12］ BREIKI F A， RIDZUAN M， GRANDHE R. Self-supervised learning for fine-grained image classification［J］. arXiv preprint arXiv:2107.13973， 2021.
［13］ SU J C， CHENG Z， MAJI S. A realistic evaluation of semi-supervised learning for fine-grained classification［C］// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2021:12966-12975.
［14］ TANG H， YUAN C C， LI Z H， et al. Learning attention-guided pyramidal features for few-shot fine-grained recognition［J］. Pattern Recognition， 2022，130. DOI: 10.1016/j.patcog.2022.108792.
［15］ FLORES C F， GONZALEZ-GARCIA A， VAN DE WEIJER J， et al. Saliency for fine-grained object recognition in domains with scarce training data［J］. Pattern Recognition， 2019，94:62-73.
［16］ ZHANG N， DONAHUE J， GIRSHICK R， et al. Part-based R-CNNs for fine-grained category detection［C］// The 13th European Conference on Computer Vision–ECCV 2014. Springer， 2014:834-849.
［17］ ZHUANG P Q， WANG Y L， QIAO Y. Learning attentive pairwise interaction for fine-grained classification［J］. Proceedings of the 2020 AAAI Conference on Artificial Intelligence， 2020，34（7）:13130-13137.
［18］江卓，吴茜，李贺武，等. 互联网端到端多路径传输跨层优化研究综述［J］. 软件学报， 2019，30（2）:302-322.
［19］ GAO Y， BEIJBOM O， ZHANG N， et al. Compact bilinear pooling［C］// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016:317-326.
［20］ LIN T Y， ROYCHOWDHURY A， MAJI S. Bilinear CNN models for fine-grained visual recognition［C］// Proceedings of 2015 IEEE International Conference on Computer Vision. IEEE， 2015:1449-1457.
［21］ ZHENG H L， FU J L， ZHA Z J， et al. Learning deep bilinear transformation for fine-grained image representation［C］// Conference on Neural Information Processing Systems. ACM， 2020:3969-4770.
［22］ CHANG D L， DING Y F， XIE J Y， et al. The devil is in the channels: Mutual-channel loss for fine-grained image classification［J］. IEEE Transactions on Image Processing， 2020，29:4683-4695.
［23］ CHOU P Y， LIN C H， KAO W C. A novel plug-in module for fine-grained visual classification［J］. arXiv preprint arXiv:2202.03822， 2022.
［24］ LAGUNAS M， IMPATA B， MARTINEZ V， et al. Transfer learning for fine-grained classification using semi-supervised learning and visual transformers［J］. arXiv preprint arXiv:2305.10018， 2023.
［25］ DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16×16 words: Transformers for image recognition at scale［J］. arXiv preprint arXiv:2010.11929， 2020.
［26］ SUN M， YUAN Y C， ZHOU F， et al. Multi-attention multi-class constraint for fine-grained image recognition［C］// Proceedings of the European Conference on Computer Vision （ECCV）. Springer， 2018:805-821.
［27］ GENG P， LU X， HU C， et al. Focusing fine-grained action by self-attention-enhanced graph neural networks with contrastive learning［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2023，33（9）:4754-4768.
［28］ LIU X Z， ZHANG L F， LI T， et al. Dual attention guided multi-scale CNN for fine-grained image classification［J］. Information Sciences， 2021，573:37-45.
［29］ HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network［J］. arXiv preprint arXiv:1503.02531， 2015.
［30］ KIM K， JI B M， YOON D， et al. Self-knowledge distillation with progressive refinement of targets［C］// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. IEEE， 2021:6567-6576.
［31］ JAISWAL A， BABU A R， ZADEH M Z， et al. A survey on contrastive self-supervised learning［J］. Technologies， 2020，9（1）. DOI: 10.3390/technologies9010002.
［32］ MAZUMDER P， SINGH P， NAMBOODIRI V P. Fair visual recognition in limited data regime using self-supervision and self-distillation［C］// Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE， 2022:3095-3103.
［33］ RAMDAN A， HERYANA A， ARISAL A， et al. Transfer learning and fine-tuning for deep learning-based tea diseases detection on small datasets［C］// 2020 International Conference on Radar， Antenna， Microwave， Electronics， and Telecommunications （ICRAMET）. IEEE， 2020:206-211.
［34］ ZHANG H， GOODFELLOW I， METAXAS D， et al. Self-attention generative adversarial networks［C］// International Conference on Machine Learning. PMLR， 2019:7354-7363.
［35］ CHEN T， KORNBLITH S， SWERSKY K， et al. Big self-supervised models are strong semi-supervised learners［J］. Advances in Neural Information Processing Systems， 2020，33:22243-22255.
［36］ WANG X， FAN H Q， TIAN Y D， et al. On the importance of asymmetry for siamese representation learning［C］// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2022:16570-16579.
［37］ CARON M， TOUVRON H， MISRA I， et al. Emerging properties in self-supervised vision Transformers［C］// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. IEEE， 2021:9650-9660.
［38］ KIM T， OH J， KIM N Y， et al. Comparing kullback-leibler divergence and mean squared error loss in knowledge distillation［J］. arXiv preprint arXiv:2105.08919， 2021.
［39］ WAH C， BRANSON S， WELINDER P， et al. The Caltechucsd Birds-200-2011 Dataset［EB/OL］. （2023-03-25）［2024-07-25］. https：//authors.library.caltech.edu/records/cvm3y-5hh21.
［40］ KRAUSE J， STARK M， DENG J， et al. 3D object representations for fine-grained categorization［C］// Proceedings of 2013 IEEE International Conference on Computer Vision Workshops. IEEE， 2013:554-561.
［41］ MAJI S， RAHTU E， KANNALA J， et al. Fine-grained visual classification of aircraft［J］. arXiv preprint arXiv:1306.5151，2013.
［42］ YU C J， ZHAO X Y， ZHENG Q， et al. Hierarchical bilinear pooling for fine-grained visual recognition［C］// Proceedings of the 15th European Conference on Computer Vision （ECCV）. ACM， 2018: 595-610.
［43］ SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2015:1-9.
［44］ SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision［C］// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016:2818-2826.
［45］ HUANG G， LIU Z， VAN DER MAATEN L， et al. Densely connected convolutional networks［C］// Proceedings of 2017 IEEE Conference on ComputerVision and Pattern Recognition. IEEE， 2017:4700-4708.
［46］ CARON M， MISRA I， MAIRAL J， et al. Unsupervised learning of visual features by contrasting cluster assignments［J］. Advances in Neural Information Processing Systems， 2020，33:9912-9924.

[1]	焦乐岩, 朱欣娟. 基于自注意力机制说话人编码器与SA-Decoder的语音克隆方法[J]. 计算机与现代化, 2025, 0(07): 69-76.
[2]	高宁波, 张晓滨. 融合自注意力的Bert-BiGRU-CRF的文本因果关系抽取[J]. 计算机与现代化, 2025, 0(07): 112-118.
[3]	梁攀如, 辛国江, 丁长松. 基于改进SegFormer的胰腺图像分割方法[J]. 计算机与现代化, 2025, 0(06): 71-78.
[4]	池碧蔚1, 孙瑞2. 基于Transformer架构的DDoS攻击检测方法[J]. 计算机与现代化, 2025, 0(05): 36-40.
[5]	程章, 刘丹, 王艳霞. 基于混合Transformer的视线估计模型[J]. 计算机与现代化, 2025, 0(04): 1-5.
[6]	郭华. 面向下游任务优化的不平衡数据重采样[J]. 计算机与现代化, 2025, 0(02): 28-32.
[7]	冯心洁, 王伟. 基于孪生特征融合网络的自然场景文本图像超分辨率方法[J]. 计算机与现代化, 2025, 0(02): 86-93.
[8]	何思达, 陈平华. 基于意图的轻量级自注意力序列推荐模型[J]. 计算机与现代化, 2024, 0(12): 1-9.
[9]	张晓东1, 白广芝1, 李敏1, 李昊洋2. 基于经验小波变换的油气井产量预测模型 [J]. 计算机与现代化, 2024, 0(12): 53-58.
[10]	祁贤, 刘大铭, 常佳鑫. 基于改进自注意力机制的多视图三维重建[J]. 计算机与现代化, 2024, 0(11): 106-112.
[11]	候聪颖, 杨文清, 王召, 程聪. 基于时频自注意力残差时序卷积网络的语音增强[J]. 计算机与现代化, 2024, 0(09): 20-24.
[12]	叶森辉, 王蕾. 结合局部自注意力和深度优化的多视图重建[J]. 计算机与现代化, 2024, 0(05): 92-98.
[13]	游嘉靖1, 2, 何月顺1, 何璘琳1, 钟海龙1, 2. 基于AHP-CNN的加密流量分类方法[J]. 计算机与现代化, 2024, 0(04): 83-87.
[14]	陈震1, 姚京辉2, 苏成悦1. 基于改进U-Net的髋关节关键点检测算法[J]. 计算机与现代化, 2024, 0(02): 15-19.
[15]	李亚平, 王军防, 余红梅, 窦一民, 肖媛, 田继林. Regformer：基于稀疏注意力的输油管道水力压降预测方法[J]. 计算机与现代化, 2024, 0(01): 59-66.