Monocular Depth Estimation Method by Aggregating Multi-dimensional Attention Features

doi:10.3969/j.issn.1006-2475.2023.06.013

Abstract

Abstract: This study is outlined to improve the precision for predicting monocular depth estimation networks and provides an in-depth analysis of the effects of multidimensional attention mechanisms on monocular depth estimation networks. The conclusions and observations are used to design a set of optimized channel and space attention blocks. Considering the convolutional neural network framework obtained based on the local plane guidance layer, a new network structure is created to fully activate the multidimensional attention mechanism through a method that is based on placing different design blocks. Furthermore, in combination with the above two measures for improvement, this study proposes a high-performance monocular depth estimation network that integrates channel and space attention features. On the KITTI Depth dataset and an NYU Depth V2 dataset, the outcomes of this study prove the effectiveness of the optimized blocks and the satisfactory performance of the proposed network through experiments. Compared with the convolutional neural network based on the local plane guidance layer, the proposed network is better in processing the overall features of images and more accurate in predicting depth information with several metrics for network evaluation improved to different degrees. The depth maps generated by the proposed network also demonstrated more data associated with the contours and details of objects.

Key words: monocular depth estimation, convolutional neural network, channel attention, spatial attention

CLC Number:

TP391

LIU Jia-jia, HU Xu-xin, YU Ping. Monocular Depth Estimation Method by Aggregating Multi-dimensional Attention Features[J]. Computer and Modernization, 2023, 0(06): 76-81.

References 30

[1]	SAXENA A, CHUNG S H, NG A Y.Learning depth from single monocular images[C]// Proceedings of the 18th International Conference on Neural Information Processing Systems. 2005:1161-1168.
[2]	EIGEN D, PUHRSCH C, FERGUS R.Depth map prediction from a single image using a multi-scale deep network[C]// Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). 2015:2650-2658.
[3]	王泉德,张松涛. 基于多尺度特征融合的单目图像深度估计[J]. 华中科技大学学报(自然科学版), 2020,48(5):7-12.
[4]	贾瑞明,李阳,李彤,等. 多层级特征融合结构的单目图像深度估计网络[J]. 计算机工程, 2020,46(12):207-214.
[5]	LI B, SHEN C, DAI Y, et al.Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015:1119-1127.
[6]	LAINA I, RUPPRECHT C, BELAGIANNIS V, et al.Deeper depth prediction with fully convolutional residual networks[C]// 2016 4th International Conference on 3D Vision (3DV). 2016:239-248.
[7]	袁建中,周武杰,潘婷,等. 基于深度卷积神经网络的道路场景深度估计[J]. 激光与光电子学进展, 2019,56(8):179-187.
[8]	GAN Y, XU X, SUN W, et al.Monocular depth estimation with affinity, vertical pooling, and label enhancement[C]// Computer Vision - ECCV 2018. 2018:232-247.
[9]	YIN W, LIU Y, SHEN C, et al.Enforcing geometric constraints of virtual normal for depth prediction[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019:5683-5692.
[10]	RANFTL R, LASINGER K, HAFNER D, et al.Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022,44(3):1623-1637.
[11]	FU H, GONG M, WANG C, et al.Deep ordinal regression network for monocular depth estimation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:2002-2011.
[12]	ALHASHIM I, WONKA P. High quality monocular depth estimation via transfer learning[J]. arXiv preprint arXiv:1812.11941, 2018.
[13]	LEE J H, HAN M, KO D W, et al.From big to small: Multi-scale local planar guidance for monocular depth estimation[J]. arXiv preprint arXiv:1907.10326, 2019.
[14]	QIAO S, ZHU Y, ADAM H, et al.ViP-DeepLab: Learning visual perception with depth-aware video panoptic segmentation[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021:3996-4007.
[15]	廖志伟,金兢,张超凡,等. 基于分层压缩激励的ASPP网络单目深度估计[J]. 图学学报, 2022,43(2):214-222.
[16]	王泉德,王奇坤,程凯,等. 强化边缘的单目图像深度估计[J]. 华中科技大学学报(自然科学版), 2022,50(3):36-42.
[17]	刘杰平,温竣文,梁亚玲. 基于多尺度注意力导向网络的单目图像深度估计[J]. 华南理工大学学报(自然科学版), 2020,48(12):52-62.
[18]	WANG Q, WU B, ZHU P, et al.ECA-Net: Efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020:11531-11539.
[19]	AICH S, VIANNEY J M U, ISLAM M A, et al. Bidirectional attention network for monocular depth estimation[C]// 2021 IEEE International Conference on Robotics and Automation (ICRA). 2021:11746-11752.
[20]	LEE S, LEE J, KIM B, et al.Patch-wise attention network for monocular depth estimation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(3):1873-1881.
[21]	CHEN W, ZHAO F, YANG D W, et al.Single-image depth perception in the wild[C]// 2016 30th Conference on Neural Information Processing System. 2016:730-738.
[22]	曾一鸣. 利用稀疏点云偏序关系的半监督单目图像深度估计[J]. 计算机辅助设计与图形学学报, 2019,31(11):2038-2046.
[23]	邹承明,胡佑璞. 引入生成对抗网络的室外场景单目深度估计[J]. 计算机工程与应用, 2021,57(6):176-183.
[24]	GARG R, BG V K, CARNEIRO G, et al.Unsupervised CNN for single view depth estimation: Geometry to the rescue[C]// Computer Vision - ECCV 2016. 2016:740-756.
[25]	GODARD C, AODHA O M, BROSTOW G J.Unsupervised monocular depth estimation with left-right consistency[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:6602-6611.
[26]	刘香凝,赵洋,王荣刚. 基于自注意力机制的多阶段无监督单目深度估计网络[J]. 信号处理, 2020,36(9):1450-1456.
[27]	CHEN L, PAPANDREOU G, KOKKINOS I, et al.DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018,40(4):834-848.
[28]	WOO S, PARK J, LEE J, et al.CBAM: Convolutional block attention module[C]// Computer Vision - ECCV 2018. 2018:3-19.
[29]	XIE S, GIRSHICK R, DOLLÄR P, et al. Aggregated residual transformations for deep neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:5987-5995.
[30]	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:4700-4708.