基于多层信息融合的实时语义分割及其在电力场景中的应用

doi:10.3969/j.issn.1006-2475.2019.08.004

计算机与现代化

基于多层信息融合的实时语义分割及其在电力场景中的应用

(国网浙江省电力有限公司信息通信分公司,浙江杭州310007)

收稿日期:2019-01-28 出版日期:2019-08-15 发布日期:2019-08-16
作者简介:周晨轶（1993-），男，浙江杭州人，助理工程师，硕士，研究方向：信息化项目管理，E-mail: macarzhou@163.com; 王文（1985-），男，浙江衢州人,工程师，本科,研究方向:信息化项目管理; 卢杉（1992-），男，浙江丽水人,助理工程师，硕士，研究方向:信息化项目管理; 徐亦白（1992-），男，山东枣庄人,助理工程师，硕士，研究方向:信息化项目管理。
基金资助:
国网浙江省电力有限公司科技项目（5211XT17000C）

Real-time Semantic Segmentation Based on Multi-scale Fusion #br# and Its Application in Electric Power Scene

(State Grid Zhejiang Electric Power Co. Ltd. Information and Communication Branch, Hangzhou 310007, China)

Received:2019-01-28 Online:2019-08-15 Published:2019-08-16

摘要/Abstract

摘要： 语义分割是计算机视觉的一项基础工作。本文提出一种新型的结合点卷积与空洞卷积的上采样结构，并基于该结构设计一个能融合多层信息的实时语义分割模型。该模型在Cityscapes数据集上的mIoU能达到72.1%，对于640×360的输入图像，运行速度能达到125 fps，同时该模型在电力场景数据集下也有很好的效果。此外，本文还将该模型移植到移动端，实现一款基于语义分割的电力场景增强现实应用。

关键词: 深度学习, 语义分割, 卷积神经网络, 电力场景

Abstract: Semantic segmentation is a basic work in computer vision. In this paper, a new upsampling structure combined point-wise convolution with dilation convolution is proposed and a real-time semantic segmentation model is designed based on this structure. The model can reach 72.1% mIoU and 125 fps running speed with the input of 640×360 on Cityscapes data set and has also good performance on a electric power scene data set. In addition, the paper transplants the model to the mobile terminal and implements an augmented reality application of electric power scene based on semantic segmentation.

Key words: deep learning, semantic segmentation, convolutional neural networks, electric power scene

中图分类号:

TP391

周晨轶，王文，卢杉，徐亦白. 基于多层信息融合的实时语义分割及其在电力场景中的应用[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2019.08.004.

ZHOU Chen-yi, WANG Wen, LU Shan， XU Yi-bai. Real-time Semantic Segmentation Based on Multi-scale Fusion #br# and Its Application in Electric Power Scene[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2019.08.004.

参考文献

［1］ SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. Computer Vision and Pottern Recognition, 2014, arXiv:1409.1556.
［2］ HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition［C］// Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［3］ EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture［C］// IEEE International Conference on Computer Vision. 2015:2650-2658.
［4］ LIU F, SHEN C, LIN G. Deep convolutional neural fields for depth estimation from a single image［C］// 2015 IEEE International Conference on Computer Vision and Pattern Recognition. 2015:5162-5170.
［5］ LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation［J］. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014,39(4):640-651.
［6］ NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation［C］// Proceedings of International Conference on Computer Vision. 2015:1520-1528.
［7］ RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// International Conference on Medical Image Computing & Computer-assisted Intervention. 2015:234-241.
［8］ YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions［C］// Proceedings of International Conference on Learning Representations. 2015.
［9］ YU F, KOLTUN V, FUNKHOUSER T. Dilated residual networks［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:636-644.
［10］ROMERA E, ALVAREZ J M, BERGASA L M, et al. ERFNet: Efficient residual factorized ConvNet for real-time semantic segmentation［J］. IEEE Transactions on Intelligent Transportation Systems, 2018,19(1):263-272.
［11］BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for scene segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(12):2481-2495.
［12］HOWARD A G, ZHU M, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications［J］. Computer Vision and Pattern Recognition, 2017, arXiv:1704.04861.
［13］CHOLLET F. Xception: Deep learning with depthwise separable convolutions［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:1251-1258.
［14］WANG P, CHEN P, YUAN Y, et al. Understanding convolution for semantic segmentation［C］// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 2018:1451-1460.
［15］ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:2881-2890.
［16］ABADI M, BARHAM P, CHEN J, et al. Tensorflow: A system for large-scale machine learning［C］// USENIX Operating System Design and Implementation. 2016,16:265-283.
［17］KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks［C］// Advances in Neural Information Processing Systems. 2012:1097-1105.
［18］CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding［C］// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2016:3213-3223.
［19］PASZKE A, CHAURASIA A, KIM S, et al. Enet: A deep neural network architecture for real-time semantic segmentation［J］. Computer Vision and Pattern Recognition, 2016, arXiv:1606.02147.
［20］TREML M, ARJONA-MEDINA J, UNTERTHINER T, et al. Speeding up semantic segmentation for autonomous driving［C］// Neural Information Processing Systems Workshop. 2016:1-5.
［21］MEHTA S, RASTEGARI M, CASPI A, et al. ESPNet: Efficient spatial pyramid of dilated convolutions for semantic segmentation［J］. Computer Vision and Pattern Recognition, 2018, arXiv:1803.06815.
［22］WANG W, PAN Z. DSNet for real-time driving scene semantic segmentation［J］. Computer Vision and Pattern Recognition, 2018, arXiv:1812.07049.
［23］ZHAO H, QI X, SHEN X, et al. Icnet for real-time semantic segmentation on high-resolution images［C］// Proceedings of the European Conference on Computer Vision. 2018:405-420.

[1]	胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.
[2]	林威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6.
[3]	周成诚, 曾庆军, 杨康, 胡家铭, 韩春伟. 基于高效通道注意力模块的运动想象脑电识别[J]. 计算机与现代化, 2023, 0(12): 19-23.
[4]	梁天恺, 黄康华, 刘凯航, 兰岚, 曾碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40.
[5]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[6]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[7]	许鸿奎, 李振业, 郭文涛, 赵京政, 郭旭斌. 基于分割的任意形状场景文本实时检测[J]. 计算机与现代化, 2023, 0(11): 95-100.
[8]	马泽宇, 叶宁, 徐康, 王甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107.
[9]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[10]	叶思佳, 魏延, 杜韩宇, 邓金枝. 结合注意力机制的HRNet图像语义分割算法[J]. 计算机与现代化, 2023, 0(10): 65-69.
[11]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[12]	沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9.
[13]	刘付琪, 张达, 宋建华, 王海东. 基于CNN-BiLSTM的液压系统故障诊断[J]. 计算机与现代化, 2023, 0(09): 10-19.
[14]	吴甜, 刘海华, 童顺延. 基于深度反馈的卷积神经网络的图像分类[J]. 计算机与现代化, 2023, 0(09): 82-86.
[15]	刘禅奕, 黄丹, 薛林雁, 王涛, 朱桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99.

基于多层信息融合的实时语义分割及其在电力场景中的应用

Real-time Semantic Segmentation Based on Multi-scale Fusion #br# and Its Application in Electric Power Scene

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价