Supermarket Fruit and Vegetable Retrieval Method Based on Deep Learning

doi:10.3969/j.issn.1006-2475.2024.04.011

Abstract

Abstract:
Abstract: In view of the problems that the current settlement method of supermarket fruits and vegetables cannot add new categories and low accuracy of small sample recognition， this paper proposes a supermarket fruits and vegetables retrieval method based on deep learning. The method obtains fruit and vegetable subjects through YOLOv4 to remove redundant background information， and extracts corresponding deep semantic features of fruit and vegetable subjects through MobileNetV3. Finally， category judgment is completed according to metric learning technology. This paper conducts experiments in accordance with the actual operation conditions of supermarkets and concludes that the method could accurately identify different fruit and vegetable categories under the condition of small samples. When the number of samples for each category is 15， the average recognition rate is about 94%， the time cost is 0.93s， and the new categories could be updated in real time. This method greatly reduces the huge labor and time cost in the actual operation of traditional supermarkets， and provides a solution for the realization of intelligence and automation in the fruit and vegetable retail industry.

Key words: Key words: image retrieval, fruit and vegetable recognition, category increase, small sample recognition

CLC Number:

TP391

GUO Zexin, ZHONG Guoyun, HE Jianfeng, ZHANG Jun. Supermarket Fruit and Vegetable Retrieval Method Based on Deep Learning[J]. Computer and Modernization, 2024, 0(04): 60-65.

References

［1］卢宏涛，张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述［J］. 数据采集与处理， 2016，31（1）:1-17.
［2］ SHAO W H， LI Y J， DIAO S F， et al. Rapid classification of Chinese quince （Chaenomeles speciosa Nakai） fruit provenance by near-infrared spectroscopy and multivariate calibration［J］. Analytical and Bioanalytical Chemistry， 2017，409:115-120.
［3］ RADI， CIPTOHADIJOYO S， LITANANDA W S， et al. Electronic nose based on partition column integrated with gas sensor for fruit identification and classification［J］. Computers and Electronics in Agriculture， 2016，121:429-435.
［4］ PARDO-MATES N， VERA A， BARBOSA S， et al. Characterization， classification and authentication of fruit-based extracts by means of HPLC-UV chromatographic fingerprints， polyphenolic profiles and chemometric methods［J］. Food Chemistry， 2017，221:29-38.
［5］ BOLLE R M， CONNELL J H， HAAS N， et al. VeggieVision: A produce recognition system［C］// Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision. IEEE， 1996:244-251.
［6］陶华伟，赵力，奚吉，等. 基于颜色及纹理特征的果蔬种类识别方法［J］. 农业工程学报， 2014，30（16）:305-311.
［7］ ZHANG Y D， DONG Z C， CHEN X Q， et al. Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation［J］. Multimedia Tools and Applications， 2019，78（3）:3613-3632.
［8］巨志勇，张泽晨. 一种局部线性编码的深度学习果蔬分类算法［J］. 小型微型计算机系统， 2020，41（4）:741-745.
［9］ KAMILARIS A， PRENAFETA-BOLDU F X. A review of the use of convolutional neural networks in agriculture［J］. The Journal of Agricultural Science， 2018，156（3）:312-322.
［10］ KOROLEV V， MITROFANOV A， KOROTCOV A， et al. Graph convolutional neural networks as "general-purpose" property predictors: The universality and limits of applicability［J］. Journal of Chemical Information and Modeling， 2020，60（1）:22-28.
［11］ GUDIVADA V N， RAGHAVAN V V. Content-based image retrieval systems［J］. Computer， 1995，28（9）:18-22.
［12］黎雯霞，赵智锋，张瑞军，等. 一种智能称重系统的设计与实现［J］. 物流技术， 2015，34（4）:167-169.
［13］ DATTA R， LI J， WANG J Z. Content-based image retrieval: Approaches and trends of the new age［C］// Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval. ACM， 2005:253-262.
［14］ HAMEED K， CHAI D， RASSAU A. A comprehensive review of fruit and vegetable classification techniques［J］. Image and Vision Computing， 2018，80:24-44.
［15］ REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN: Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017，39（6）:1137-1149.
［16］ BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection［J］. arXiv preprint arXiv:2004.10934， 2020.
［17］ HE K M， ZHANG X Y， REN S Q， et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2014，37（9）:1904-1916.
［18］ LIU S， QI L， QIN H F， et al. Path aggregation network for instance segmentation［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2018:8759-8768.
［19］ LUO W J， LI Y J， URTASUN R， et al. Understanding the effective receptive field in deep convolutional neural networks［C］// Proceedings of the 30th International Conference on Neural Information Processing Systems. ACM， 2016:4905-4913.
［20］ NEYSHABUR B， BHOJANAPALLI S， MCALLESTER D， et al. Exploring generalization in deep learning［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM， 2017:5949-5958.
［21］ ZHAO G D， WU Y， CHEN F Q， et al. Effective feature selection using feature vector graph for classification［J］. Neurocomputing， 2015，151:376-389.
［22］ HOWARD A， SANDLER M， CHEN B， et al. Searching for MobileNetV3［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. IEEE， 2019:1314-1324.
［23］ GUO Y M， LIU Y， OERLEMANS A， et al. Deep learning for visual understanding: A review［J］. Neurocomputing，2016，187:27-48.
［24］ KUMARI A， SHARMA N. A review on convolutional neural networks for skin lesion classification［C］// Proceedings of the 2021 2nd International Conference on Secure Cyber Computing and Communications （ICSCCC）. IEEE， 2021:186-191.
［25］ KAYA M， BILGE H S. Deep metric learning: A survey［J］. Symmetry， 2019，11（9）. DOI: 10.3390/sym11091066.
［26］ XU X Y. Research on the design of the intelligent transportation cycling system based on the Internet of Things technology simulating［C］// Proceedings of the 2016 2nd International Conference on Electronics： Network and Computer Engineering （ICENCE 2016）. Atlantis Press， 2016:66-69.
［27］张万红，朱元骏. 套袋苹果影像的分割方法研究［J］. 北方园艺， 2017（8）:6-9.
［28］苏欣，赖复尧，余容平，等. 基于多视角模板匹配的零件图像检索方法［J］. 机械制造与自动化， 2023，52（1）:222-225.
［29］金柱璋，方旭源，黄彦慧，等. 基于深度度量学习的卫星云图像检索［J］. 光电工程， 2022，49（4）:15-25.
［30］ LIN C H， CHEN R T， CHAN Y K. A smart content-based image retrieval system based on color and texture feature［J］. Image and Vision Computing， 2009，27（6）:658-665.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	ZHANG Kun1, ZHANG Yongwei1, WU Yongcheng1, ZHANG Xiaowen2, ZHAI Shichen2. An LLM-based Method for Automatic Construction of Equipment Failure Knowledge Graphs [J]. Computer and Modernization, 2024, 0(11): 46-53.
[3]	ZHANG Yu1, 2, LI Jing1, 2, MA Ming1, 2, WANG Zhongxiang1, 2, SUN Yan1, 2. YOLOLW: A Novel Lightweight Object Detection Model [J]. Computer and Modernization, 2024, 0(11): 91-98.
[4]	DU Mengjun1, LI Ang1, TONG Jun1, QIAN Jin1, KANG Kai1, WANG Ruoding1, JIN Wenxing2. Power Information Data Fusion Model Based on Improved Extreme Learning Algorithm [J]. Computer and Modernization, 2024, 0(10): 61-64.
[5]	JIAO Yikai1, 2, ZHU Xinjuan1, 2. Label Recommendation Methods for Public Cultural Resources [J]. Computer and Modernization, 2024, 0(10): 107-112.
[6]	YANG Yufeng1, 2, XIA Xiaoyun2, CHEN Zefeng3, LIAO Weizhi2, LI Jiwu2. Dung Beetle Optimization Algorithm Integrating Multiple Strategies for Take-out Order Distribution Route Optimization [J]. Computer and Modernization, 2024, 0(09): 25-32.
[7]	MA Yu, YANG Yong, REN Ge, Palidan Tuerxun. Automated Essay Scoring Method Based on GCN and Fine Tuned BERT [J]. Computer and Modernization, 2024, 0(09): 33-37.
[8]	LIU Wenliang1, WU Fei1, HE Deming1, ZHAO Weiwei2, PAN Jianhong3. Text Clustering Method for Fragmented Reply Based on Dissimilarity Matrix [J]. Computer and Modernization, 2024, 0(09): 56-60.
[9]	GAO Meng, ZENG Xianwen. Improved Pelican Optimization Algorithm Based on Circle Mapping and#br# Adaptive t-Distribution Mutation [J]. Computer and Modernization, 2024, 0(09): 69-73.
[10]	YU Chenxi, GU Lin. Recognition and Warning of Elevator Abnormal Behavior Based on Human Skeleton [J]. Computer and Modernization, 2024, 0(09): 114-120.
[11]	WANG Yan, CONG Xin, ZI Lingling. Combining Knowledge Tracing and Graph Convolution for Knowledge Concept#br# Recommendation [J]. Computer and Modernization, 2024, 0(08): 17-23.
[12]	FU Shugang1, 2, 3. Multi-object Tracking of UAV Based on Improved YOLOX and New Data Association Method [J]. Computer and Modernization, 2024, 0(08): 59-66.
[13]	WEI Jiakun, WANG Jiarun. Survey on Gesture Recognition and Interaction [J]. Computer and Modernization, 2024, 0(08): 67-76.
[14]	WANG Tao1, 2, HUANG Dan1, 2, LIU Chanyi1, 2, ZHU Tao1, 2. Vehicle Detection in UAV Image Based on YOLOv5s [J]. Computer and Modernization, 2024, 0(08): 108-113.
[15]	YANG Jiang1, SUN Xiaomei1, XU Tao2. Stock Price Prediction Based on Business Content to Construct Stock Association Relationships [J]. Computer and Modernization, 2024, 0(07): 21-25.