Wheat Image Recognition Based on Global Self-attention

Abstract

Abstract: In the actual application scenario, it is very challenging to identify wheat diseases and pests by image recognition. Compared with the previous methods based solely on convolutional neural network (CNN), the method of converting wheat images into a series of visual languages and recognizing wheat from a global perspective is more feasible and practical. The use of convolutional visual Transformers (CVT) to solve wheat recognition is divided into two links. First, two feature maps generated by two-branch CNN are used to realize attentional selective fusion (ASF). ASF obtains different information by fusing multiple features and global-local attention, and projects it into a series of visual languages. Secondly, inspired by the success of Transformers in natural language processing, global self-attention is used to model the relationship between these visual languages. Compared with classical classification networks LeNet-5, ResNet-18, VGG-16 and EfficientNet, CVT improves the recognition rate, and this method has good generalization ability.

Key words: wheat recognition, global-local attention, Transformer, global self-attention

HE Chen-xi, WANG Zheng-yong, QING Lin-bo, HE Xiao-hai, WU Xiao-qiang. Wheat Image Recognition Based on Global Self-attention[J]. Computer and Modernization, 2022, 0(04): 38-44.

References ［30］

［1］	ZAYAS I, POMERANZ, LAI F S. Discrimination between Arthur and Arkan wheats by image analysis［J］. Cereal Chemistry, 1985,62(6):478-480.
［2］	THOMSON W H, POMERANZ Y. Classification of wheat kernels using three-dimensional image analysis［J］. Cereal Chemistry, 1991,68(4):357-361.
［3］	NEETHIRAJAN S, JAYAS D S, WHITE N D G. Detection of sprouted wheat kernels using soft X-ray image analysis［J］. Journal of Food Engineering, 2007,81(3):509-513.
［4］	陈丰农. 基于机器视觉的小麦并肩杂与不完善粒动态实时检测研究［D］. 杭州:浙江大学, 2012.
［5］	曹婷翠,何小海,董德良,等. 基于CNN深度模型的小麦不完善粒识别［J］. 现代计算机(专业版), 2017(36):9-14.
［6］	陈文根. 基于深度学习的小麦图谱特征技术研究［D］. 郑州:河南工业大学, 2018.
［7］	祝诗平,卓佳鑫,黄华,等. 基于CNN的小麦籽粒完整性图像检测系统［J］. 农业机械学报, 2020,51(5):36-42.
［8］	LI Y, ZENG J B, SHAN S G, et al. Occlusion aware facial expression recognition using CNN with attention mechanism［J］. IEEE Transactions on Image Processing, 2019,28(5):2439-2450.
［9］	WANG Z N, ZENG F W, LIU S C, et al. OAENet: Oriented attention ensemble for accurate facial expression recognition［J］. Pattern Recognition, 2021,112. DOI: 10.1016/j.patcog.2020.107694.
［10］	WEN P Z, DING Y, WEN Y Y, et al. Facial expression recognition method based on convolution neural network combining attention mechanism［C］// Proceedings of the 2020 International Conference on Artificial Intelligence and Security. 2020:136-147.
［11］	ZHENG W M. Multi-view facial expression recognition based on group sparse reduced-rank regression［J］. IEEE Transactions on Affective Computing, 2014,5(1):71-85.
［12］	LIU Y Y, ZENG J B, SHAN S G, et al. Multi-channel pose-aware convolution neural networks for multi-view facial expression recognition［C］// Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). 2018:458-465.
［13］	DIN N U, JAVED K, BAE S, et al. A novel GAN-based network for unmasking of masked face［J］. IEEE Access, 2020,8:44276-44287.〖HJ1.18mm〗
［14］	SHEN X B, SUN Q S, YUAN Y H. A unified multiset canonical correlation analysis framework based on graph embedding for multiple feature extraction［J］. Neurocomputing, 2015,148:397-408.
［15］	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［16］	PENG C, ZHANG X Y, YU G, et al. Large kernel matters: Improve semantic segmentation by global convolutional network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:1743-1751.
［17］	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:6000-6010.
［18］	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with Transformers［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020:213-229.
［19］	师小燕,万韬阮,汤汶,等. 姿态估计算法在古建筑室内环境中的应用研究［J］. 计算机与现代化, 2012(11):214-216.
［20］	MAHAPATRA D, KUANAR S, BOZORGTABAR B, et al. Self-supervised learning of inter-label geometric relationships for Gleason grade segmentation［M］// Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health. Springer, 2021:57-67.
［21］	WANG Y Q, XU Z L, WANG X L, et al. End-to-end video instance segmentation with transformers［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:8737-8746.
［22］	PARK S H, LEE G, SEO J, et al. Diverse and admissible trajectory forecasting through multimodal context understanding［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020:282-298.
［23］	WU B C, XU C F, DAI X L, et al. Visual Transformers: Token-based image representation and processing for computer vision［J］. arXiv preprint arXiv:2006.03677, 2020.
［24］	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge［J］. International Journal of Computer Vision, 2015,115(3):211-252.
［25］	WANG W H, XIE E Z, LI X, et al. Pyramid vision Transformer: A versatile backbone for dense prediction without convolutions［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021:548-558.
［26］	YOU J Y, KORHONEN J. Transformer for image quality assessment［C］// Proceedings of the 2021 IEEE International Conference on Image Processing (ICIP). 2021:1389-1393.
［27］	LECUN Y. LeNet-5, convolutional neural networks［DB/OL］. ［2021-09-07］. http://yann.lecun.com/exdb/lenet.
［28］	ALIPPI C, DISABATO S, ROVERI M. Moving convolutional neural networks to embedded systems: The AlexNet and VGG-16 case［C］// Proceedings of the 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN). 2018:212-223.
［29］	MAHBOD A, SCHAEFER G, WANG C L, et al. Skin lesion classification using hybrid deep neural networks［C］// Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2019:1229-1233.
［30］	TAN M X, LE Q V. EfficientNet: Rethinking model scaling for convolutional neural networks［C］// Proceedings of the 36th International Conference on Machine Learning. 2019:6105-6114.

[1]	LI Ya-ping, WANG Jun-fang, YU Hong-mei, DOU Yi-min, XIAO Yuan, TIAN Ji-lin. Regformer： Hydraulic Prediction Model of Oil Pipeline Based on GS-XGBoost [J]. Computer and Modernization, 2024, 0(01): 59-66.
[2]	FU Hong-lin, ZHANG Tai-hong, YANG Ya-ting, Aizimaiti Aiwanier, MA Bo. Scenes Text Modification Network for Uyghur Based on Generative Adversarial Network [J]. Computer and Modernization, 2024, 0(01): 41-46.
[3]	XU Ye-tong, GENG Xin-zhe, ZHAO Wei-qiang, ZHANG Yue, NING Hai-long, LEI Tao. A Remote Sensing Image Change Detection Model Based on CNN-Transformer Hybrid Structure [J]. Computer and Modernization, 2023, 0(07): 79-85.
[4]	LIU Jing, CHEN Jin-guang. Image Caption Generation Method Based on Channel Attention and Transformer [J]. Computer and Modernization, 2023, 0(05): 8-12.
[5]	WANG Hong-yi, KONG Mei-mei, XU Rong-qing. Flame Detection Algorithm Based on Improved YOLOV5 [J]. Computer and Modernization, 2023, 0(01): 103-107.
[6]	LIN Zhen-xian, QU Jia-xin, LUO Liang. Enhanced Image Caption Based on Improved Transformer_decoder [J]. Computer and Modernization, 2023, 0(01): 7-12.
[7]	LI Xin, REN De-jun, REN Qiu-lin, CAO Lin-jie, YAN Zong-yi. Detection and Location Method for Hub Weld Based on Retinanet [J]. Computer and Modernization, 2022, 0(09): 60-67.
[8]	CHEN Tong, ZHOU Deng-wen. Multistage-transformer Large-factor Network: Reference-based Super-resolution [J]. Computer and Modernization, 2022, 0(08): 121-126.
[9]	LIN Fan-qin, LI Ming-ming, GUO Hong. Review on Fault Diagnosis Technology of Transformer [J]. Computer and Modernization, 2022, 0(03): 116-126.
[10]	FENG Ru-jia, ZHANG Hai-jun, PAN Wei-min. Microblog Rumor Detection Based on Sentiment Analysis and Transformer Model [J]. Computer and Modernization, 2021, 0(10): 1-7.
[11]	LI Xin-ye, ZHANG Cheng-qiang, ZHOU Xiong-tu, GUO Tai-liang, ZHANG Yong-ai. Multi-scene Fusion Algorithm for Fine-grained Image Caption [J]. Computer and Modernization, 2021, 0(09): 1-6.
[12]	YANG Jia-xue, PENG Guo-zheng, HAN Li-xin. Video Click-through Rate Prediction Model Based on Users Dynamic Interests [J]. Computer and Modernization, 2021, 0(03): 82-87.
[13]	GUO Jian-long1, XIONG Shan1, LI Xiao-ying2, QI Yan-wei2, WU Cheng-kai3. Virtual Reality Training System Based on Main Transformer Fire Emergency Drill [J]. Computer and Modernization, 2020, 0(05): 75-.
[14]	SHAO Kai-li;WANG Hong-yun. Research on System of Power Transformer Fault Diagnosis Based on WSN and Data Fusion Technology [J]. Computer and Modernization, 2012, 203(7): 171-175.
[15]	WANG Shao-jie;HUA Ze-xi;WANG Li. Design of Online Monitoring System for Thermal Characteristics of Oil Immersed Transformer [J]. Computer and Modernization, 2011, 1(8): 110-113.