Real-time Semantic Segmentation Based on Multi-scale Fusion #br# and Its Application in Electric Power Scene

doi:10.3969/j.issn.1006-2475.2019.08.004

Abstract

Abstract: Semantic segmentation is a basic work in computer vision. In this paper, a new upsampling structure combined point-wise convolution with dilation convolution is proposed and a real-time semantic segmentation model is designed based on this structure. The model can reach 72.1% mIoU and 125 fps running speed with the input of 640×360 on Cityscapes data set and has also good performance on a electric power scene data set. In addition, the paper transplants the model to the mobile terminal and implements an augmented reality application of electric power scene based on semantic segmentation.

Key words: deep learning, semantic segmentation, convolutional neural networks, electric power scene

CLC Number:

TP391

ZHOU Chen-yi, WANG Wen, LU Shan， XU Yi-bai. Real-time Semantic Segmentation Based on Multi-scale Fusion #br# and Its Application in Electric Power Scene[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2019.08.004.

References

［1］ SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. Computer Vision and Pottern Recognition, 2014, arXiv:1409.1556.
［2］ HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition［C］// Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［3］ EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture［C］// IEEE International Conference on Computer Vision. 2015:2650-2658.
［4］ LIU F, SHEN C, LIN G. Deep convolutional neural fields for depth estimation from a single image［C］// 2015 IEEE International Conference on Computer Vision and Pattern Recognition. 2015:5162-5170.
［5］ LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation［J］. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014,39(4):640-651.
［6］ NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation［C］// Proceedings of International Conference on Computer Vision. 2015:1520-1528.
［7］ RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// International Conference on Medical Image Computing & Computer-assisted Intervention. 2015:234-241.
［8］ YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions［C］// Proceedings of International Conference on Learning Representations. 2015.
［9］ YU F, KOLTUN V, FUNKHOUSER T. Dilated residual networks［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:636-644.
［10］ROMERA E, ALVAREZ J M, BERGASA L M, et al. ERFNet: Efficient residual factorized ConvNet for real-time semantic segmentation［J］. IEEE Transactions on Intelligent Transportation Systems, 2018,19(1):263-272.
［11］BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for scene segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(12):2481-2495.
［12］HOWARD A G, ZHU M, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications［J］. Computer Vision and Pattern Recognition, 2017, arXiv:1704.04861.
［13］CHOLLET F. Xception: Deep learning with depthwise separable convolutions［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:1251-1258.
［14］WANG P, CHEN P, YUAN Y, et al. Understanding convolution for semantic segmentation［C］// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 2018:1451-1460.
［15］ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:2881-2890.
［16］ABADI M, BARHAM P, CHEN J, et al. Tensorflow: A system for large-scale machine learning［C］// USENIX Operating System Design and Implementation. 2016,16:265-283.
［17］KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks［C］// Advances in Neural Information Processing Systems. 2012:1097-1105.
［18］CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016:3213-3223.
［19］PASZKE A, CHAURASIA A, KIM S, et al. Enet: A deep neural network architecture for real-time semantic segmentation［J］. Computer Vision and Pattern Recognition, 2016, arXiv:1606.02147.
［20］TREML M, ARJONA-MEDINA J, UNTERTHINER T, et al. Speeding up semantic segmentation for autonomous driving［C］// Neural Information Processing Systems Workshop. 2016:1-5.
［21］MEHTA S, RASTEGARI M, CASPI A, et al. ESPNet: Efficient spatial pyramid of dilated convolutions for semantic segmentation［J］. Computer Vision and Pattern Recognition, 2018, arXiv:1803.06815.
［22］WANG W, PAN Z. DSNet for real-time driving scene semantic segmentation［J］. Computer Vision and Pattern Recognition, 2018, arXiv:1812.07049.
［23］ZHAO H, QI X, SHEN X, et al. Icnet for real-time semantic segmentation on high-resolution images［C］// Proceedings of the European Conference on Computer Vision. 2018:405-420.

[1]	HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation [J]. Computer and Modernization, 2024, 0(01): 74-79.
[2]	LIN Wei. Incremental News Recommendation Method Based on Self-supervised Learning and Data Replay [J]. Computer and Modernization, 2023, 0(12): 1-6.
[3]	LIANG Tian-kai, HUANG Kang-hua, LIU Kai-hang, LAN Lan, ZENG Bi. Deep Federated Image Classification Method Based on Bilateral Homomorphic Encryption [J]. Computer and Modernization, 2023, 0(12): 36-40.
[4]	QIU Kai-xing, FENG Guang. A Multi-label Image Classification Model Based on Dual Feature Attention [J]. Computer and Modernization, 2023, 0(12): 41-47.
[5]	ZHANG Bo-quan, MAI Hai-peng, CHEN Jia-min, Pang Jin-ju. White Matter Hyperintensities Segmentation Based on High Gray Value#br# Attention Mechanism [J]. Computer and Modernization, 2023, 0(12): 67-75.
[6]	XU Hong-kui, LI Zhen-ye, GUO Wen-tao, ZHAO Jing-zheng, GUO Xu-bin. Real-time Detection of Arbitrary Shape Scene Text Based on Segmentation [J]. Computer and Modernization, 2023, 0(11): 95-100.
[7]	LI Yan-man, WANG Bi-heng, ZHAO Ling-yan. Safety Helmet Detection Based on Lightweight YOLOv5 [J]. Computer and Modernization, 2023, 0(10): 59-64.
[8]	YE Si-jia, WEI Yan, DU Han-yu, DENG Jin-zhi. HRNet Image Semantic Segmentation Algorithm Combined with Attention Mechanism [J]. Computer and Modernization, 2023, 0(10): 65-69.
[9]	LI Shi-da, XIANG Jian-wen. A Weakened Joint Reinforcement Method to Improve Robustness of Image Recognition Models [J]. Computer and Modernization, 2023, 0(10): 70-76.
[10]	SHEN Jia-wei, LU Yi-ming, CHEN Xiao-yi, QIAN Mei-ling, LU Wei-zhong, . Review of Research on Human Behavior Detection Methods Based on Deep Learning [J]. Computer and Modernization, 2023, 0(09): 1-9.
[11]	LIU Fu-qi, ZHANG Da, SONG Jian-hua, WANG Hai-dong. Fault Diagnosis of Hydraulic Systems Based on CNN-BiLSTM [J]. Computer and Modernization, 2023, 0(09): 10-19.
[12]	LIU Chan-yi, HUANG Dan, XUE Lin-yan, WANG Tao, ZHU Tao, . COVID-19 X-ray Classification Based on Improved Efficientnet Network [J]. Computer and Modernization, 2023, 0(09): 94-99.
[13]	MA Guo-xiang, YANG Ling-fei, YAN Chuan-bo, ZHANG Zhi-hao, SUN Bing, WANG Xiao-rong. Ultrasonic Image Diagnosis of Hepatic Echinococcosis Based on Deep DenseNet Network [J]. Computer and Modernization, 2023, 0(09): 100-104.
[14]	NONG Hao-cheng, REN De-jun, REN Qiu-lin, LIU Peng-li, HUANG De-cheng. Surface Anomaly Detection Algorithm of Flexible Plastic Packaging Based on Improved ConvNeXt [J]. Computer and Modernization, 2023, 0(08): 12-17.
[15]	LIU Xu, ZHA Ke-ke. An Environmental Target Recognition Method for Airport Special Vehicle Operation [J]. Computer and Modernization, 2023, 0(08): 18-24.