结合注意力机制的HRNet图像语义分割算法

doi:10.3969/j.issn.1006-2475.2023.10.010

摘要/Abstract

摘要： 目前主流的语义分割算法中依然存在小尺寸目标丢失、分割不精确等问题，针对这些问题本文基于HRNet网络模型进行改进，融入注意力机制生成更有效的特征图，对于原模型中低分辨率图像直接向高分辨率图像融合而产生的特征图细节不足的问题，提出多级上采样机制，让不同分辨率图像之间的融合方式更平滑从而得到更好的融合效果，同时使用深度可分离卷积减少模型的参数。本文模型全程保持了图像较高的分辨率，保留了特征图的空间信息，提升了对小尺寸目标的分割效果。在PASCAL VOC2012增强版数据集上的mIoU值达到80.87%，和原模型相比，精度提升了1.54个百分点。

关键词: 关键词：图像语义分割, 注意力机制, 高分辨率, 深度可分离卷积

Abstract: Abstract: The current mainstream semantic segmentation algorithms still have problems such as loss of small-sized objects and inaccurate segmentation. In response to these problems， this paper improves the HRNet network model and integrates the attention mechanism to generate more effective feature maps. To address the problem of insufficient detail of the feature map caused by the direct fusion of the low resolution images to the high-resolution images in the original model， a multi-level upsampling mechanism is proposed to make the fusion between images of different resolutions smoother to achieve better fusion results， and the depth separable convolution is used to reduce the parameters of the model. The model in this article maintains a high resolution of the image throughout the entire process. The spatial information of the feature map is improved， and the segmentation effect of small-sized objects is improved. The mIoU value on the PASCAL VOC2012 enhanced dataset reaches 80.87%， and the accuracy is improved by 1.54 percentage points compared with the original model.

Key words: Key words: image semantic segmentation, attention mechanism, high resolution, depthwise separable convolution

中图分类号:

TP391

叶思佳, 魏延, 杜韩宇, 邓金枝. 结合注意力机制的HRNet图像语义分割算法[J]. 计算机与现代化, 2023, 0(10): 65-69.

YE Si-jia, WEI Yan, DU Han-yu, DENG Jin-zhi. HRNet Image Semantic Segmentation Algorithm Combined with Attention Mechanism[J]. Computer and Modernization, 2023, 0(10): 65-69.

参考文献［27］

［1］	田萱，王亮，丁琪. 基于深度学习的图像语义分割方法综述［J］. 软件学报， 2019，30（2）:440-468.
［2］	YU H S， YANG Z G， TAN L， et al. Methods and datasets on semantic segmentation: A review［J］. Neurocomputing， 2018，304:82-103.
［3］	QIU C P， MOU L C， SCHMITT M， et al. Local climate zone-based urban land cover classification from multi-seasonal sentinel-2 images with a recurrent residual network［J］. ISPRS Journal of Photogrammetry and Remote Sensing， 2019，154:151-162.
［4］	MODY P. Semantic segmentation: Wiki， applications and resources［EB/OL］. （2018-12-06）［2020-05-24］. https://www.kdnuggets.com/2018/10/semantic-segmentation-wiki
	-applications-resources.html.
［5］	SUN L， YANG K L， HU X X， et al. Real-time fusion network for RGB-D semantic segmentation incorporating unexpected obstacle detection for road-driving images［J］. IEEE Robotics and Automation Letters， 2020，5（4）:5558-5565.
［6］	王龙飞，严春满. 道路场景语义分割综述［J/OL］. 激光与光电子学进展:1-25（2020-10-15）［2021-03-02］. http://kns.cnki.net/kcms/detail/31.1690.TN.20201015.0915.002.html.
［7］	JIN Q G， MENG Z P， PHAM T D， et al. DUNet: A deformable network for retinal vessel segmentation［J］. Knowledge
	-Based Systems， 2019，178:149-162.
［8］	贾园园. 基于深度学习和形变模型的乳腺癌全扫描切片图像语义分割［D］. 武汉:华中科技大学， 2019.
［9］	OTSU N. A threshold selection method from gray-level histograms［J］. IEEE Transactions on Systems， Man， and Cybernetics， 1979，9（1）:62-66.
［10］	YANG L， WU X Y， ZHAO D W， et al. An improved Prewitt algorithm for edge detection based on noised image［C］// Proceedings of the 2011 4th International Congress on Image and Signal Processing. 2011，3:1197-1200.
［11］	COATES A， NG A Y. Learning feature representations with K-means［M］// Neural Networks: Tricks of the Trade. Springer， 2012:561-580.
［12］	TANG M， GORELICK L， VEKSLER O， et al. GrabCut in one cut［C］// Proceedings of the 2013 IEEE International Conference on Computer Vision. 2013:1769-1776.
［13］	SHELHAMER E， LONG J， DARRELL T. Fully convolutional networks for semantic segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017，39（4）:640-651.
［14］	RONNEBERGER O， FISCHER P， BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015:234-241.
［15］	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017，39（12）:2481-2495.
［16］	ZHAO H S， SHI J P， QI X J， et al. Pyramid scene parsing network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017:6230-6239.
［17］	LI H C， XIONG P F， AN J， et al. Pyramid attention network for semantic segmentation［J］. arXiv preprint arXiv:1805.10180， 2018.
［18］	YU F， KOLTUN V， FUNKHOUSER T. Dilated residual networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017:636-644.
［19］	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation［J］. arXiv preprint arXiv:1706.05587， 2017.
［20］	CHEN L C， PAPANDREOU G， KOKKINOS I， et al. DeepLab: Semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018，40（4）:834-848.
［21］	CHEN L C， ZHU Y K， PAPANDREOU G， et al. Encoder-decoder with atrous separable convolution for semantic image segmentation［C］// Proceedings of the 2018 European Conference on Computer Vision （ECCV）. 2018:833-851.
［22］	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019:5686-5696.
［23］	WANG F， JIANG M Q， QIAN C， et al. Residual attention network for image classification［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017:6450-6458.
［24］	FU J， LIU J， TIAN H J， et al. Dual attention network for scene segmentation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019:3141-3149.
［25］	CAO J X， CHEN Q， GUO J， et al. Attention-guided context feature pyramid network for object detection［J］. arXiv preprint arXiv:2005.11475， 2020.
［26］	CHOLLET F. Xception: Deep learning with depthwise separable convolutions［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017:1800-1807.
［27］	EVERINGHAM M， ESLAMI S M A， VAN GOOL L， et al. The Pascal visual object classes challenge: A retrospective［J］. International Journal of Computer Vision， 2015，111（1）:98-136.

[1]	王秋忆, 周浩, 郑婷婷. 改进RetinaNet的电力设备目标检测方法[J]. 计算机与现代化, 2024, 0(01): 47-52.
[2]	林启钊, 彭志平, 郭棉, 崔得龙. 基于双向多步预测的炉管温度场重构方法[J]. 计算机与现代化, 2024, 0(01): 53-58.
[3]	李亚平, 王军防, 余红梅, 窦一民, 肖媛, 田继林. Regformer：基于稀疏注意力的输油管道水力压降预测方法[J]. 计算机与现代化, 2024, 0(01): 59-66.
[4]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[5]	张浩洋, 尹梓名, 乐珺怡, 沈达聪, 束翌俊, 杨自逸, 孔祥勇, 龚伟. 3D-SPRNet: 一种基于并行解码器和双注意力机制的胆囊癌分割模型[J]. 计算机与现代化, 2023, 0(12): 59-66.
[6]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[7]	宁娟, 周庆华, 曾小为. 改进YOLOv7算法在西林瓶轧盖缺陷检测中的应用[J]. 计算机与现代化, 2023, 0(12): 82-86.
[8]	王宇航, 董宝良, 公超, 尚真真, 姚康宁. 基于意图识别的空中群目标动态威胁评估[J]. 计算机与现代化, 2023, 0(12): 100-104.
[9]	罗明杰, 冯开平. 基于沙漏结构与注意力机制的轻量级人脸表情识别方法[J]. 计算机与现代化, 2023, 0(11): 89-94.
[10]	张嘉琪, 徐啟蕾. 基于NAM-YOLO网络的苹果缺陷检测算法[J]. 计算机与现代化, 2023, 0(10): 53-58.
[11]	陈嘉敏, 张伯泉, 麦海鹏. 基于特征融合的海马体分割[J]. 计算机与现代化, 2023, 0(08): 1-6.
[12]	刘续, 查可可. 一种用于机场特种车辆作业的环境目标识别方法[J]. 计算机与现代化, 2023, 0(08): 18-24.
[13]	王鸿, 葛红. 基于注意力机制和语义相似度的跨模态哈希检索[J]. 计算机与现代化, 2023, 0(08): 44-53.
[14]	欧阳飞, 吴旭, 向东升. 基于改进YOLOX的垃圾分类检测方法[J]. 计算机与现代化, 2023, 0(08): 68-73.
[15]	崔少国, 张岗, 王奥迪. 基于感知注意力的深度交叉网络推荐模型[J]. 计算机与现代化, 2023, 0(07): 54-60.