计算机视觉下的果实目标检测算法综述

摘要/Abstract

摘要： 基于计算机视觉的果实目标检测识别是目标检测、计算机视觉、农业机器人等多学科的重要交叉研究课题，在智慧农业、农业现代化、自动采摘机器人等领域，具有重要的理论研究意义和实际应用价值。随着深度学习在图像处理领域中广泛应用并取得良好效果，计算机视觉技术结合深度学习方法的果实目标检测识别算法逐渐成为主流。本文介绍基于计算机视觉的果实目标检测识别的任务、难点和发展现状，以及2类基于深度学习方法的果实目标检测识别算法，最后介绍用于算法模型训练学习的公开数据集与评价模型性能的评价指标，且对当前果实目标检测识别存在的问题和未来可能的发展方向进行讨论。

关键词: 计算机视觉, 深度学习, 果实检测, 目标检测

Abstract: Fruit target detection and recognition based on computer vision is an important cross-disciplinary research topic of target detection, computer vision, agricultural robots, etc. It has important theoretical research significance and practical application value in the fields of smart agriculture, agricultural modernization, and automatic picking robots. As deep learning is widely used in the field of image processing and has achieved good results, fruit target detection and recognition algorithms combining computer vision technology with deep learning methods gradually become the mainstream. This article introduces the tasks, difficulties and development status of fruit target detection and recognition based on computer vision, as well as two types of fruit target detection and recognition algorithms based on deep learning methods. Finally, the public data set used for the training and learning of the algorithm model and the evaluation index for evaluating the performance of the model are introduced, and the current problems in the detection and recognition of fruit targets and the possible future development directions are discussed.

Key words: computer vision, deep learning, fruit detection, target detection

李伟强, 王东, 宁政通, 卢明亮, 覃鹏飞. 计算机视觉下的果实目标检测算法综述[J]. 计算机与现代化, 2022, 0(06): 87-95.

LI Wei-qiang, WANG Dong, NING Zheng-tong, LU Ming-liang, QIN Peng-fei. Survey of Fruit Object Detection Algorithms in Computer Vision[J]. Computer and Modernization, 2022, 0(06): 87-95.

参考文献［59］

［1］	徐铭辰,牛媛媛,余永昌. 果蔬采摘机器人研究综述［J］. 安徽农业科学, 2014,42(31):11024-11027.
［2］	程祥云,宋欣. 果蔬采摘机器人视觉系统研究综述［J］. 浙江农业科学, 2019,60(3):490-493.
［3］	高文硕,宋卫东,王教领,等. 果蔬菌采摘机械研究综述［J］. 中国农机化学报, 2020,41(10):9-15.
［4］	蒋焕煜,彭永石,申川,等. 基于双目立体视觉技术的成熟番茄识别与定位［J］. 农业工程学报, 2008,24(8):279-283.
［5］	LOWE D G. Distinctive image features from scale-invariant keypoints［J］. International Journal of Computer Vision, 2004,60(2):91-110.
［6］	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection［C］// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). 2005:886-893.
［7］	BURGES C J C. A tutorial on support vector machines for pattern recognition［J］. Data Mining and Knowledge Discovery, 1998,2(2):121-167.
［8］	FREUND Y, SCHAPIRE R E. A decision-theoretic generalization of on-line learning and an application to boosting［J］. Journal of Computer and System Sciences, 1997,55(1):119-139.
［9］	JI W, ZHAO D, CHENG F Y, et al. Automatic recognition vision system guided for apple harvesting robot［J］. Computers and Electrical Engineering, 2012,38(5):1186-1195.
［10］	SI Y S, LIU G, FENG J. Location of apples in trees using stereoscopic vision［J］. Computers and Electronics in Agriculture, 2015,112:68-74.
［11］	陶华伟,赵力,奚吉,等. 基于颜色及纹理特征的果蔬种类识别方法［J］. 农业工程学报, 2014,30(16):305-311.
［12］	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks［C］// 2012 Advances in Neural Information Processing Systems. 2012:1097-1105.
［13］	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:580-587.
［14］	UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object regonition［J］. International Journal of Computer Vision, 2013,104(2):154-171.
［15］	穆龙涛,高宗斌,崔永杰,等. 基于改进AlexNet的广域复杂环境下遮挡猕猴桃目标识别［J］. 农业机械学报, 2019,50(10):24-34.
［16］	GIRSHICK R. Fast R-CNN［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. 2015:1440-1448.
［17］	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556, 2014.
［18］	周云成,许童羽,邓寒冰,等. 基于双卷积链Fast R-CNN的番茄关键器官识别方法［J］. 沈阳农业大学学报, 2018,49(1):65-74.
［19］	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015:91-99.
［20］	朱旭,马淏,姬江涛,等. 基于Faster R-CNN的蓝莓冠层果实检测识别分析［J］. 南方农业学报, 2020,51(6):1493-1501.
［21］	HE K M, GKIOXARI G, DOLLR P, et al. Mask R-CNN［C］// 2017 IEEE International Conference on Computer Vision (ICCV). 2017:2980-2988.
［22］	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:8759-8768.
［23］	CAO Z, SIMON T, WEI S E, et al. Realtime multi-person 2D pose estimation using part affinity fields［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:7291-7299.
［24］	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation［C］// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015:3431-3440.
［25］	宁政通,罗陆锋,廖嘉欣,等. 基于深度学习的葡萄果梗识别与最优采摘定位［J］. 农业机械学报, 2021,37(9):222-229.
［26］	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:779-788.
［27］	REDMON J, FARHADI A. YOLO9000: Better, faster, stronger［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:6517-6525.
［28］	刘芳,刘玉坤,林森,等. 基于改进型YOLO的复杂环境下番茄果实快速识别方法［J］. 农业机械学报, 2020,51(6):229-237
［29］	IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift［C］// Proceedings of the 32nd International Conference on International Conference on Machine Learning. 2015:448-456.
［30］	REDMON J, FARHADI A. YOLOv3: An increment improvement［J］. arXiv preprint arXiv:1804.02767, 2018.
［31］	薛月菊,黄宁,涂淑琴,等. 未成熟芒果的改进YOLOv2识别方法［J］. 农业工程学报, 2018,34(7):173-179.
［32］	HOSMER D W, LEMESHOW S. Applied Logistic Regression［M］. John Wiley & Sons, 2013.
［33］	SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-resNet and the impact of residal connections on learning［J］. arXiv preprint arXiv:1602.07261, 2016.
［34］	LIN T Y, DOLLR P, GIRSHICK R, et al. Feature pyramid networks for object detection［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:936-944.
［35］	JAIN A K. Data clustering: 50 years beyond K-means［J］. Pattern Recognition Letters, 2010,31(8):651-666.
［36］	BOCHKOVSKIY A, WANG C Y, MARK LIAO H Y. YOLOv4: Optimal speed and accuracy of object detection［J］. arXiv preprint arXiv:2004.10934, 2020.
［37］	武星,齐泽宇,王龙军,等. 基于轻量化YOLOv3卷积神经网络的苹果检测方法［J］. 农业机械学报, 2020,51(8):17-25.
［38］	HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(9):1904-1916.
［39］	LI C, DENG C, LI N, et al. Self-supervised adversarial hashing networks for cross-modal retrieval［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. 2018:4242-4251.
［40］	张晴晖,孔德肖,李俊萩,等. 基于逆运动学降维求解与YOLOv4的果实采摘系统设计［J/OL］. 农业机械学报:1-15［2021-07-30］. http://kns.cnki.net/kcms/detail/11.1964.S.20210526.1723.018.html.
［41］	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shotmultibox detector［C］// 2016 European Conference on Computer Vision. 2016:21-37.
［42］	李善军,胡定一,高淑敏,等. 基于改进SSD的柑橘实时分类检测［J］. 农业工程学报, 2019,35(24):307-313.
［43］	彭红星,黄博,邵园园,等. 自然环境下多类水果采摘目标识别的通用改进SSD模型［J］. 农业工程学报, 2018,34(16):155-162.
［44］	岳有军,田博凯,王红君,等. 基于改进Mask RCNN的复杂环境下苹果检测研究［J］. 中国农机化学报, 2019,40(10):128-134.
［45］	赵德安,吴任迪,刘晓洋,等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位［J］. 农业工程学报, 2019,35(3):164-173.
［46］	朱旭,马淏,姬江涛,等. 基于Faster R-CNN的蓝莓冠层果实检测识别分析［J］. 南方农业学报, 2020,51(6):1493-1501.
［47］	刘芳,刘玉坤,林森,等. 基于改进型YOLO的复杂环境下番茄果实快速识别方法［J］. 农业机械学报, 2020,51(6):229-237.
［48］	曾镜源,洪添胜,杨洲. 基于实例分割的柚子姿态识别与定位研究［J］. 河南农业大学学报, 2021,55(2):287-294.
［49］	ZHENG Y Y, KONG J L, JIN X B, et al. CropDeep: The crop vision dataset for deep-learning-based classification and detection in precision agriculture［J］. Sensors, 2019,19(5). DOI: 10.3390/s19051058.
［50］	HNI N, ROY P, ISLER V. MinneApple: A benchmark dataset for apple detection and segmentation［J］. arXiv preprint arXiv:1909.06441, 2019.
［51］	GEN-MOLA J, VILAPLANA V, ROSELL-POLO J R, et al. Multi-modal deep learning for fruit detection using RGB-D cameras and their radiometric capabilities［J］. Computers and Electronics in Agriculture, 2019,162:689-698.
［52］	FERLEZ J. Methods for analysis of research related data in the IST-world application［D］. University of Ljubljana, 2007.
［53］	HOREA M, MIHAI O. Fruit recognition from images using deep learning［J］. Acta Universitatis Sapientiae Informatica, 2018,10(1):26-42.
［54］	WALTNER G, SCHWARZ M, LADSTTTER S, et al. Personalized dietary self-management using mobile vision-based assistance［M］// New Trends in Image Analysis and Processing-ICIAP 2017. 2017:385-393.
［55］	EVERINGHAM M, VAN GOOL L, WILLIAMS C K I, et al. The PASCAL visual object classes (VOC) challenge［J］. International Journal of Computer Vision, 2010,88(2):303-338.
［56］	EVERINGHAM M, ALI ESLAMI S M, VAN GOOL L, et al. The PASCAL visual object classes challenge：A retrospective［J］. International Journal of Computer Vision, 2015,111(1):98-136.
［57］	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge［J］. International Journal of Computer Vision, 2015,115(3):211-252.
［58］	HOIEM D, CHODPATHUMWAN Y, DAI Q Y. Diagnosing error in object detectors［C］// Proceedings of the 12th European Conference on Computer Vision. 2012:340-353.
［59］	陈兆凡,赵春阳,李博. 一种改进IoU损失的边框回归损失函数［J］. 计算机应用研究, 2020,37(S2):293-296.

[1]	王秋忆, 周浩, 郑婷婷. 改进RetinaNet的电力设备目标检测方法[J]. 计算机与现代化, 2024, 0(01): 47-52.
[2]	胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.
[3]	林威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6.
[4]	梁天恺, 黄康华, 刘凯航, 兰岚, 曾碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40.
[5]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[6]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[7]	马泽宇, 叶宁, 徐康, 王甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107.
[8]	欧嘉城, 曾安, 金亮. 基于CP-YOLOX的冷冻电镜图像蛋白质目标检测算法[J]. 计算机与现代化, 2023, 0(11): 113-119.
[9]	闫子贤, 董宝良, 唐思谜. 针对复杂背景下低分辨率舰船目标的改进YOLOv7算法[J]. 计算机与现代化, 2023, 0(11): 120-126.
[10]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[11]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[12]	沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9.
[13]	顾成伟, 丁勇, 李登华. 基于计算机视觉的工业厂区人员安全警戒系统[J]. 计算机与现代化, 2023, 0(09): 20-26.
[14]	刘禅奕, 黄丹, 薛林雁, 王涛, 朱桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99.
[15]	马国祥, 杨凌菲, 严传波, 张志豪, 孙彬, 王晓荣. 基于深度DenseNet网络的肝包虫病超声影像诊断方法[J]. 计算机与现代化, 2023, 0(09): 100-104.