Visual Grasping of Robot Arm in Chemical Experiment Scene

Abstract

Abstract: Aiming at the problem that it is difficult for the depth camera to detect the distance of transparent objects such as test tubes in chemical experiment scenes, and the robot arm is difficult to obtain the three-dimensional coordinates of the chemical test tubes in space, an improved deep learning algorithm YOLOv3 Tiny is proposed to detect the sticker labels on the test tube to obtain the three-dimensional coordinates of transparent chemical test tube. In view of the problem that different chemical test tubes cannot be classified, a deep learning algorithm CTPN+BLSTM+CTC Loss is proposed to identify the text information on the label to classify the test tubes. In this paper, a depth camera, a monocular camera and a six-axis robotic arm equipped with a ROS system are used as the experimental platform to train a chemical label detection model and a text detection recognition model on TensorFlow. The ROS system on the Raspberry Pi equipped with the robotic arm is used to perform Python programming and make grasping test about the chemical test tubes with different chemical labels. The results show that this method has a high recognition rate and positioning accuracy for the transparent test tubes with labels. It can realize the robotic arm grabbing chemical test tubes containing different substances.

Key words: robotic arm, visual capture, transparent object detection, handwriting recognition

MA Xing-lu, ZHANG Xing-qiang, WANG Tao. Visual Grasping of Robot Arm in Chemical Experiment Scene[J]. Computer and Modernization, 2021, 0(10): 88-93.

References

［1］ LIU L, OUYANG W L, WANG X G, et al. Deep learning for generic object detection: A survey［J］. International Journal of Computer Vision, 2020,128(2):261-318.
［2］ CHEN G H, WANG J Y, ZHANG A J. Transparent object detection and location based on RGB-D camera［C］// Proceedings of the 16th International Conference on Metrology and Properties of Engineering Surfaces. 2017. DOI: 10.1088/1742-6596/1183/1/012011.
［3］ SEIB V, BARTHEN A, MAROHN P, et al. Friend or foe: Exploiting sensor failures for transparent object localization and classification［C］// Proceedings of the 2016 International Conference on Robotics and Machine Vision. 2016. DOI: 10.1117/12.2266255.
［4］ SAJJAN S, MOORE M, PAN M, et al. Clear grasp: 3D shape estimation of transparent objects for manipulation［C］// Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA). 2020:3634-3642.
［5］ REDMON J, FARHADI A. YOLOv3: An incremental improvement［J］. arXiv preprint arXiv:1804.02767, 2018.
［6］方路平,何杭江,周国民. 目标检测算法研究综述［J］. 计算机工程与应用, 2018,54(13):11-18.
［7］杜学丹,蔡莹皓,鲁涛,等. 一种基于深度学习的机械臂抓取方法［J］. 机器人, 2017,39(6):820-828.

［8］ REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:779-788.

［9］ DHILLON A, VERMA G K. Convolutional neural network: A review of models, methodologies and applications to object detection［J］. Progress in Artificial Intelligence, 2020,9(2):85-112.
［10］鞠默然,罗海波,王仲博,等. 改进的YOLO V3算法及其在小目标检测中的应用［J］. 光学学报, 2019,39(7):245-252.
［11］MISRA D. Mish: A self regularized non-monotonic neural activation function［J］. arXiv preprint arXiv:1908.08681, 2019.
［12］LIU Y L, JIN L W. Deep matching prior network: Toward tighter multi-oriented text detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:3454-3461.
［13］SHI B G, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:3482-3490.
［14］DAI Y C, HUANG Z, GAO Y T, et al. Fused text segmentation networks for multi-oriented scene text detection［C］// Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR). 2018:3604-3609.
［15］HU H, ZHANG C Q, LUO Y X, et al. WordSup: Exploiting word annotations for character based text detection［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV). 2017:4950-4959.
［16］TIAN Z, HUANG W L, HE T, et al. Detecting text in natural image with connectionist text proposal network ［C］// Proceedings of the 2016 European Conference on Computer Vision (ECCV). 2016:56-72.
［17］SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556, 2014.
［18］王润民,桑农,丁丁,等. 自然场景图像中的文本检测综述［J］. 自动化学报, 2018,44(12):2113-2141.
［19］HOCHREITER S, SCHMIDHUBER J. Long short-term memory［J］. Neural Computation, 1997,9(8):1735-1780.
［20］GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18(5-6):602-610.
［21］GRAVES A, FERNANDEZ S, GOMEZ F, et al. Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks［C］// Proceedings of the 23rd International Conference on Machine learning. 2006:369-376.
［22］金连文,钟卓耀,杨钊,等. 深度学习在手写汉字识别中的应用综述［J］. 自动化学报, 2016,42(8):1125-1141.
［23］孟韶南,梁雁冰,师恒. 基于ROS平台的六自由度机械臂运动规划［J］. 上海交通大学学报, 2016,50(S1):94-97.

[1]	ZHANG Hui, YU Hou-yun, LI Ke-bin. Surface Quality Visual Inspection of Turbine Shell Based on Robotic Arm [J]. Computer and Modernization, 2020, 0(07): 121-126.
[2]	SHANG Xue-lian, LIANG Chuan-jun. A Balance Algorithm Between Handwriting Error and User Effort [J]. Computer and Modernization, 2015, 0(9): 50-56.