Few-shot Object Detection via Learnable Memory Feature Pyramid Network

doi:10.3969/j.issn.1006-2475.2023.12.002

Abstract

Abstract: Abstract: At present， it is difficult to obtain the data of some industry application scenarios， and the problem of few shot has become an important factor restricting the application and promotion of deep learning technology. In this paper， few shot method is adopted to improve the performance of the model in the absence of data and reduce the dependence of the deep learning model on data， and few-shot object detection via learnable memory feature pyramid network is proposed to retain cleaner multi-scale feature information for classifier prediction. With the help of the adaptive feature fusion module， the network can choose the emphasis ratio among the features of different levels to maximize the retention of discriminant feature information of different scales. At the same time， we also add a retrospective feature alignment module to alleviate the feature confusion effect introduced by stacking feature layers. The experimental results show that the model performance can be effectively improved by overcoming the dependence on data， and the improved model can surpass other existing models of the same type in the COCO dataset and VOC dataset. In particular， when the prior parameter k is set to 5 in VOC dataset， nAP50 increases by 4.8 to 44.7； when the prior parameter k is set to 30 in COCO dataset， nAP50 increases by 4.0 to 29.4.

Key words: Key words： few shot, adaptive fusion, feature alignment, feature pyramid network

CLC Number:

TP391

XIA Qian-han, HE Sheng-huang, WU Yuan-qing, ZHAO Le-le. Few-shot Object Detection via Learnable Memory Feature Pyramid Network[J]. Computer and Modernization, 2023, 0(12): 7-13.

References

［1］ OGUZ C， VU N T. Few-shot learning for slot tagging with attentive relational network［C］// Proceedings of the 2021 EACL Association for Computational Linguistics. 2021：1566-1572.
［2］吴晗，张志龙，李楚为，等. 小样本红外图像的样本扩增与目标检测算法［J］. 控制理论与应用， 2021，38（9）：1477-1485.
［3］ ZHOU J， ZHENG Y N， TANG J， et al. FlipDA： Effective and robust data augmentation for few-shot learning［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022：8646-8665.
［4］张振伟，郝建国，黄健，等. 小样本图像目标检测研究综述［J］. 计算机工程与应用， 2022，58（5）：1-11.
［5］徐培，赵雪专，唐红强，等. 基于两阶段投票的小样本目标检测方法［J］. 计算机应用， 2014，34（4）：1126-1129.
［6］彭云聪，秦小林，张力戈，等. 面向图像分类的小样本学习算法综述［J］. 计算机科学， 2022，49（5）：1-9.
［7］ KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［C］// Proceedings of the 2012 International Conference on Neural Information Processing Systems. 2012：1097-1105.
［8］ XIAO Y， MARLET R. Few-shot object detection and viewpoint estimation for objects in the wild［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020：192-210.
［9］ QIAO L M， ZHAO Y X， LI Z Y， et al. DeFRCN： Decoupled faster R-CNN for few-shot object detection［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021：8661-8670.
［10］ RUSU A A， RAO D， SYGNOWSKI J， et al. Meta-learning with latent embedding optimization［C］// Proceedings of the 2019 International Conference on Learning Representations. 2019：43-49.
［11］ SUN Q R， LIU Y Y， CHUA T S， et al. Meta-transfer learning for few-shot learning［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：403-412.
［12］ CHEN W Y， LIU Y C， KIRA Z， et al. A closer look at few-shot classification［C］// Proceedings of the 2019 International Conference on Learning Representations. 2019：1873-1881.
［13］ HAN G X， HE Y C， HUANG S Y， et al. Query adaptive few-shot object detection with heterogeneous graph convolutional networks［C］// Proceedings of the 2021 IEEE International Conference on Computer Vision （ICCV）. 2021：3243-3252.
［14］ LI B W， WANG C， REDDY P， et al. AirDet： Few-shot detection without fine-tuning for autonomous exploration［C］// Proceedings of the 2022 European Conference on Computer Vision. 2022：427-444.
［15］ Sun B， Li B H， Cai S C， et al. FSCE： Few-shot object detection via contrastive proposal encoding［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2021：7348-7358.
［16］林润超，黄荣，董爱华. 基于注意力机制和元特征二次重加权的小样本目标检测［J］. 计算机应用， 2022，42（10）：3025-3032.
［17］ FU J L， ZHENG H L， MEI T. Look closer to see better： Recurrent attention convolutional neural network for fine-grained image recognition［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：4476-4484.
［18］邓杰航，郭文权，陈汉杰，等. 融合多尺度多头自注意力和在线难例挖掘的小样本硅藻检测［J］. 计算机应用， 2022，42（8）：2593-2600.
［19］ LIN T Y， DOLLAR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：936-944.
［20］ CHEN H， WANG Y L， WANG G Y， et al. LSTD： A low-shot transfer detector for object detection［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018：2836-2843.
［21］ KANG B Y， LIU Z， WANG X， et al. Few-shot object detection via feature reweighting［C］// Proceedings of the 2019 IEEE International Conference on Computer Vision （ICCV）. 2019：8419-8428.
［22］ HAN G X， HUANG S Y， MA J W， et al. Meta faster R-CNN： Towards accurate few-shot object detection with attentive feature alignment［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. 2022：780-789.
［23］ WANG X， HUANG T E， DARRELL T， et al. Frustratingly simple few-shot object detection［C］// Proceedings of the 37th International Conference on Machine Learning. 2020：9919-9928.
［24］ WU J X， LIU S T， HUANG D， et al. Multi-scale positive sample refinement for few-shot object detection［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020：456-472.
［25］ YANG Z， WANG Y L， CHEN X Y， et al. Context-transformer： Tackling object confusion for few-shot detection［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020：12653-12660.
［26］ EVERINGHAM M， VAN GOOL L， WILLIAMS C K I， et al. The PASCAL visual object classes （VOC） challenge［J］. International Journal of Computer Vision， 2010，88（2）：303-338.
［27］ LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： Common objects in context［C］// Proceedings of the 2014 European Conference on Computer Vision. 2014：740-755.
［28］ REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017，39（6）：1137-1149.
［29］ GUO C X， FAN B， ZHANG Q， et al. AugFPN： Improving multi-scale feature learning for object detection［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020：12592-12601.
［30］ LUO Y H， CAO X， ZHANG J T et al. CE-FPN： Enhancing channel information for object detection［J］. Multimedia Tools and Applications， 2022，81（21）：30685-30704.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	ZHANG Kun1, ZHANG Yongwei1, WU Yongcheng1, ZHANG Xiaowen2, ZHAI Shichen2. An LLM-based Method for Automatic Construction of Equipment Failure Knowledge Graphs [J]. Computer and Modernization, 2024, 0(11): 46-53.
[3]	ZHANG Yu1, 2, LI Jing1, 2, MA Ming1, 2, WANG Zhongxiang1, 2, SUN Yan1, 2. YOLOLW: A Novel Lightweight Object Detection Model [J]. Computer and Modernization, 2024, 0(11): 91-98.
[4]	DU Mengjun1, LI Ang1, TONG Jun1, QIAN Jin1, KANG Kai1, WANG Ruoding1, JIN Wenxing2. Power Information Data Fusion Model Based on Improved Extreme Learning Algorithm [J]. Computer and Modernization, 2024, 0(10): 61-64.
[5]	JIAO Yikai1, 2, ZHU Xinjuan1, 2. Label Recommendation Methods for Public Cultural Resources [J]. Computer and Modernization, 2024, 0(10): 107-112.
[6]	YANG Yufeng1, 2, XIA Xiaoyun2, CHEN Zefeng3, LIAO Weizhi2, LI Jiwu2. Dung Beetle Optimization Algorithm Integrating Multiple Strategies for Take-out Order Distribution Route Optimization [J]. Computer and Modernization, 2024, 0(09): 25-32.
[7]	MA Yu, YANG Yong, REN Ge, Palidan Tuerxun. Automated Essay Scoring Method Based on GCN and Fine Tuned BERT [J]. Computer and Modernization, 2024, 0(09): 33-37.
[8]	LIU Wenliang1, WU Fei1, HE Deming1, ZHAO Weiwei2, PAN Jianhong3. Text Clustering Method for Fragmented Reply Based on Dissimilarity Matrix [J]. Computer and Modernization, 2024, 0(09): 56-60.
[9]	GAO Meng, ZENG Xianwen. Improved Pelican Optimization Algorithm Based on Circle Mapping and#br# Adaptive t-Distribution Mutation [J]. Computer and Modernization, 2024, 0(09): 69-73.
[10]	YU Chenxi, GU Lin. Recognition and Warning of Elevator Abnormal Behavior Based on Human Skeleton [J]. Computer and Modernization, 2024, 0(09): 114-120.
[11]	WANG Yan, CONG Xin, ZI Lingling. Combining Knowledge Tracing and Graph Convolution for Knowledge Concept#br# Recommendation [J]. Computer and Modernization, 2024, 0(08): 17-23.
[12]	FU Shugang1, 2, 3. Multi-object Tracking of UAV Based on Improved YOLOX and New Data Association Method [J]. Computer and Modernization, 2024, 0(08): 59-66.
[13]	WEI Jiakun, WANG Jiarun. Survey on Gesture Recognition and Interaction [J]. Computer and Modernization, 2024, 0(08): 67-76.
[14]	WANG Tao1, 2, HUANG Dan1, 2, LIU Chanyi1, 2, ZHU Tao1, 2. Vehicle Detection in UAV Image Based on YOLOv5s [J]. Computer and Modernization, 2024, 0(08): 108-113.
[15]	YANG Jiang1, SUN Xiaomei1, XU Tao2. Stock Price Prediction Based on Business Content to Construct Stock Association Relationships [J]. Computer and Modernization, 2024, 0(07): 21-25.