基于机械臂的化学物品定位与分类

计算机与现代化 ›› 2021, Vol. 0 ›› Issue (10): 88-93.

基于机械臂的化学物品定位与分类

(青岛科技大学信息科学技术学院,山东青岛266061)

出版日期:2021-10-14 发布日期:2021-10-14
作者简介:马兴录(1970—),男,山东临沂人,副教授,硕士生导师,硕士,研究方向:嵌入式系统与自动化控制,E-mail: qdmxl@163.com; 通信作者:张兴强(1995—),男,山东淄博人,硕士研究生,研究方向:嵌入式系统与软件开发,E-mail: 953233624@qq.com; 王涛(1995—),男,山东临沂人,硕士研究生,研究方向:嵌入式系统与软件开发,E-mail: 861177678@qq.com。
基金资助:
国家重点研发计划项目(2017YFB1400903); 山东省重点研发计划项目(2017GSF218088)

Visual Grasping of Robot Arm in Chemical Experiment Scene

(College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061, China)

Online:2021-10-14 Published:2021-10-14

摘要/Abstract

摘要： 针对化学实验场景下深度相机难以探测试管等透明物体距离，继而引起机械臂难以获取化学试管在空间中的三维坐标的问题，提出通过改进的深度学习算法YOLOv3 Tiny检测试管上的贴纸标签以获取透明化学试管的三维空间坐标；针对不同化学试管无法分类的问题，提出通过深度学习算法CTPN+BLSTM+CTC Loss识别标签上的文字信息对试管进行分类。本文采用深度相机、单目相机与搭载ROS系统的六轴机械臂为实验平台，在TensorFlow上训练化学标签检测模型与文字检测识别模型。通过在机械臂搭载的树莓派上的ROS系统进行Python编程对贴有不同的化学标签的化学试管进行抓取实验，结果显示该方法对贴有标签的透明试管具有较高的识别率及定位准确率，可以实现机械臂抓取装有不同物质的化学试管。

关键词: 机械臂, 视觉捕获, 透明目标检测, 手写文字识别

Abstract: Aiming at the problem that it is difficult for the depth camera to detect the distance of transparent objects such as test tubes in chemical experiment scenes, and the robot arm is difficult to obtain the three-dimensional coordinates of the chemical test tubes in space, an improved deep learning algorithm YOLOv3 Tiny is proposed to detect the sticker labels on the test tube to obtain the three-dimensional coordinates of transparent chemical test tube. In view of the problem that different chemical test tubes cannot be classified, a deep learning algorithm CTPN+BLSTM+CTC Loss is proposed to identify the text information on the label to classify the test tubes. In this paper, a depth camera, a monocular camera and a six-axis robotic arm equipped with a ROS system are used as the experimental platform to train a chemical label detection model and a text detection recognition model on TensorFlow. The ROS system on the Raspberry Pi equipped with the robotic arm is used to perform Python programming and make grasping test about the chemical test tubes with different chemical labels. The results show that this method has a high recognition rate and positioning accuracy for the transparent test tubes with labels. It can realize the robotic arm grabbing chemical test tubes containing different substances.

Key words: robotic arm, visual capture, transparent object detection, handwriting recognition

马兴录, 张兴强, 王涛. 基于机械臂的化学物品定位与分类[J]. 计算机与现代化, 2021, 0(10): 88-93.

MA Xing-lu, ZHANG Xing-qiang, WANG Tao. Visual Grasping of Robot Arm in Chemical Experiment Scene[J]. Computer and Modernization, 2021, 0(10): 88-93.

参考文献

［1］ LIU L, OUYANG W L, WANG X G, et al. Deep learning for generic object detection: A survey［J］. International Journal of Computer Vision, 2020,128(2):261-318.
［2］ CHEN G H, WANG J Y, ZHANG A J. Transparent object detection and location based on RGB-D camera［C］// Proceedings of the 16th International Conference on Metrology and Properties of Engineering Surfaces. 2017. DOI: 10.1088/1742-6596/1183/1/012011.
［3］ SEIB V, BARTHEN A, MAROHN P, et al. Friend or foe: Exploiting sensor failures for transparent object localization and classification［C］// Proceedings of the 2016 International Conference on Robotics and Machine Vision. 2016. DOI: 10.1117/12.2266255.
［4］ SAJJAN S, MOORE M, PAN M, et al. Clear grasp: 3D shape estimation of transparent objects for manipulation［C］// Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA). 2020:3634-3642.
［5］ REDMON J, FARHADI A. YOLOv3: An incremental improvement［J］. arXiv preprint arXiv:1804.02767, 2018.
［6］方路平,何杭江,周国民. 目标检测算法研究综述［J］. 计算机工程与应用, 2018,54(13):11-18.
［7］杜学丹,蔡莹皓,鲁涛,等. 一种基于深度学习的机械臂抓取方法［J］. 机器人, 2017,39(6):820-828.

［8］ REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:779-788.

［9］ DHILLON A, VERMA G K. Convolutional neural network: A review of models, methodologies and applications to object detection［J］. Progress in Artificial Intelligence, 2020,9(2):85-112.
［10］鞠默然,罗海波,王仲博,等. 改进的YOLO V3算法及其在小目标检测中的应用［J］. 光学学报, 2019,39(7):245-252.
［11］MISRA D. Mish: A self regularized non-monotonic neural activation function［J］. arXiv preprint arXiv:1908.08681, 2019.
［12］LIU Y L, JIN L W. Deep matching prior network: Toward tighter multi-oriented text detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:3454-3461.
［13］SHI B G, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:3482-3490.
［14］DAI Y C, HUANG Z, GAO Y T, et al. Fused text segmentation networks for multi-oriented scene text detection［C］// Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR). 2018:3604-3609.
［15］HU H, ZHANG C Q, LUO Y X, et al. WordSup: Exploiting word annotations for character based text detection［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). 2017:4950-4959.
［16］TIAN Z, HUANG W L, HE T, et al. Detecting text in natural image with connectionist text proposal network ［C］// Proceedings of the 2016 European Conference on Computer Vision (ECCV). 2016:56-72.
［17］SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556, 2014.
［18］王润民,桑农,丁丁,等. 自然场景图像中的文本检测综述［J］. 自动化学报, 2018,44(12):2113-2141.
［19］HOCHREITER S, SCHMIDHUBER J. Long short-term memory［J］. Neural Computation, 1997,9(8):1735-1780.
［20］GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18(5-6):602-610.
［21］GRAVES A, FERNANDEZ S, GOMEZ F, et al. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks［C］// Proceedings of the 23rd International Conference on Machine learning. 2006:369-376.
［22］金连文,钟卓耀,杨钊,等. 深度学习在手写汉字识别中的应用综述［J］. 自动化学报, 2016,42(8):1125-1141.
［23］孟韶南,梁雁冰,师恒. 基于ROS平台的六自由度机械臂运动规划［J］. 上海交通大学学报, 2016,50(S1):94-97.

[1]	宋涛涛, 李艳萍, 李洪港, 韩春雪. 基于改进变结构趋近律的机械臂滑模控制系统[J]. 计算机与现代化, 2023, 0(12): 14-18.
[2]	张辉, 余厚云, 李克斌. 基于机械臂的涡轮壳零件表面质量视觉检测[J]. 计算机与现代化, 2020, 0(07): 121-126.
[3]	马兴录，何爱欣，李莹莹. 桌面型机械臂的机器视觉应用框架研究[J]. 计算机与现代化, 2019, 0(11): 64-.
[4]	张金龙，邹裕龙，杨斌，姚灿杰，郑耀宗. 基于二阶段双向搜索的解魔方机器人系统[J]. 计算机与现代化, 2019, 0(02): 82-.
[5]	韩丽丽,王奇志,杨永刚. 机械臂抓取行为规划研究综述[J]. 计算机与现代化, 2018, 0(09): 11-.
[6]	袁文翀;王从庆;武和雷;刘秀超. 自由浮动空间冗余度机械臂避奇异规化的混沌运动抑制控制[J]. 计算机与现代化, 2012, 198(2): 56-60.