基于无监督域适应的室外点云语义分割

doi:10.3969/j.issn.1006-2475.2024.01.012

摘要/Abstract

摘要： 摘要：为处理室外大规模场景中语义分割网络训练需求数据量过大的问题，提出一种基于无监督域自适应的点云语义分割方法。该方法使用改进的RandLA-Net以SPTLS3D真实世界数据集的少量点云作为目标对象进行语义分割。模型在SensatUrban数据集上完成分割网络的预训练，通过缩小源域和目标域之间的域差距来完成模型的迁移。RandLA-Net编码过程会缺失原始点云全局特征，因此本文提出一种额外获取全局信息加入网络解码的方法。此外，为增强差异化信息的获取，RandLA-Net的局部注意力模块权值改为根据各点的特征和其邻域的平均特征的差值。实验显示，该网络在SemanticKITTI数据集上的平均交并比精度达到54.3%，在Semantic3D上的平均交并比精度达到了71.91%。预训练好的模型经过微调后平均交并比精度达到了80.05%，比直接训练的效果好8.83个百分点。

关键词: 关键词：点云语义分割, 无监督领域自适应, 迁移学习, 微调, 深度学习

Abstract:

Abstract: An unsupervised domain adaptation for LiDAR semantic segmentation method is proposed to deal with the problem of excessive data required for semantic segmentation network training in outdoor large-scale scenes. The method uses a modified RandLA-Net for semantic segmentation using a small number of point clouds from the SPTLS3D’s real world data as target objects. The model finishes the pre-training of the segmentation network on SensatUrban， and completes the transfer task by minimizing the domain gap between the source and target domains. The RandLA-Net losses the global features of the original point cloud in the encoding process， so an additional method of obtaining global information to join the network decoding is proposed. In addition， for getting the differentiated information， the weights of the local attention module of RandLA-Net is changed to use the difference between the features of each point and the average features of its neighbors. The experiments show that the mean intersection over union of the network are 54.3% on SemanticKITTI and 71.91% on Semantic3D. The mIoU of the pre-trained network after fine-tuning are 80.05%， which is 8.83 percentage points better than training directly.

Key words: Key words: point cloud semantic segmentation, unsupervised domain adaptation, transfer learning, fine-tune, deep learning

中图分类号:

TP393

胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.

HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation[J]. Computer and Modernization, 2024, 0(01): 74-79.

参考文献［28］

［1］	BOULCH A， LE SAUX B， AUDEBERT N. Unstructured point cloud semantic labeling using deep segmentation networks［C］// Proceedings of the Workshop on 3D Object Retrieval. 2017:17-24.
［2］	LAWIN F J， DANELLJAN M， TOSTEBERG P， et al. Deep projective 3D semantic segmentation［C］// Proceedings of 17th International Conference on Computer Analysis of Images and Patterns. 2017:95-107.
［3］	KIM M， ILYAS N， KIM K. AMSASeg: An attention-based multi-scale atrous convolutional neural network for real-time object segmentation from 3D point cloud［J］. IEEE Access， 2021，9:70789-70796.
［4］	CHENG R， RAZANI R， TAGHAVI E， et al. （AF）2-S3Net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network［C］. // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:12542-12551.
［5］	ZHU X G， ZHOU H， WANG T， et al. Cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:9934-9943.
［6］	杨晓文，李静，韩燮，等. 基于八叉树的卷积神经网络三维模型分割［J］. 计算机工程与设计， 2020，41（9）:2663-2669.
［7］	QI C R， SU H， KAICHUN M， et al. PointNet: Deep learning on point sets for 3D classification and segmentation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:77-85.
［8］	QI C R， YI L， SU H， et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space［C］// Proceedings of the 31st Annual Conference on Neural Information Processing Systems. 2017:5105-5114.
［9］	THOMAS H， QI C R， DESCHAUD J E， et al. KPconv: Flexible and deformable convolution for point clouds［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 2019:6410-6419.
［10］	MA X， QIN C， YOU H X， et al. Rethinking network design and local geometry in point cloud: A simple residual MLP framework［J］. arXiv preprint arXiv:2202.07123， 2022.
［11］	XU J Y， ZHANG R X， DOU J， et al. RPVNet: A deep and efficient range-point-voxel fusion network for LiDAR point cloud segmentation［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. 2021:16004-16013.
［12］	ZHOU H， ZHU X G， SONG X， et al. Cylinder3D: An effective 3D framework for driving-scene lidar semantic segmentation ［J］. arXiv preprint arXiv:2008.01550， 2020.
［13］	LANDRIEU L， SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:4558-4567.
［14］	HU Q Y， YANG B， XIE L H， et al. RandlA-Net: Efficient semantic segmentation of large-scale point clouds［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:11105-11114.
［15］	SHUAI H， XU X， LIU Q S. Backward attentive fusing network with local aggregation classifier for 3D point cloud semantic segmentation［J］. IEEE Transactions on Image Processing， 2021，30:4973-4984.
［16］	苏鸣方，胡立坤，黄润辉. 基于上下文注意力的室外点云语义分割方法［J］. 计算机工程， 2023，49（3）:248-256.
［17］	CHEN M D， HU Q Y， YU Z F， et al. STPLS3D: A large-scale synthetic and real aerial photogrammetry 3D point cloud dataset［J］. arXiv preprint arXiv:2203.09065， 2022.
［18］	WU B C， ZHOU X Y， ZHAO S C， et al. SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud［C］// 2019 International Conference on Robotics and Automation. 2019:4376-4382.
［19］	XIAO A R， HUANG J X， GUAN D Y， et al. Transfer learning from synthetic to real LiDAR point cloud for semantic segmentation［C］. Proceedings of the AAAI Conference on Artificial Intelligence. 2022，36（3）:2795-2803.
［20］	HOFFMAN J， WANG D Q， YU F S， et al. FCNs in the wild: Pixel-level adversarial and constraint-based adaptation［J］. arXiv preprint arXiv:1612.02649， 2016.
［21］	WU B C， WAN A L， YUE X Y， et al. SqueezeSeg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud［C］// 2018 IEEE International Conference on Robotics and Automation. 2018:1887-1893.
［22］	ZHAO S C， WANG Y Z， LI B， et al. ePointDA: An end-to-end simulation-to-real domain adaptation framework for LiDAR point cloud segmentation［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2021，35（4）:3500-3509.
［23］	HU Q Y， YANG B， KHALID S， et al. Towards semantic segmentation of urban-scale 3D point clouds: A dataset， benchmarks and challenges ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:4975-4985.
［24］	RONNEBERGER O， FISCHER P， BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015:234-241.
［25］	LI Y Y， BU R， SUN M C， et al. PointCNN: Convolution on X-transformed points［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018:828-838.
［26］	SUN B C， SAENKO K. Deep coral: Correlation alignment for deep domain adaptation［C］// European Conference on Computer Vision. 2016:443-450.
［27］	HACKEL T， SAVINOV N， LADICKY L， et al. Semantic3D.net: A new large-scale point cloud classification benchmark［J］. arXiv preprint arXiv:1704.03847， 2017.
［28］	BEHLEY J， GARBADE M， MILIOTO A， et al. SemanticKITTI: A dataset for semantic scene understanding of lidar sequences［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 2019:9296-9306.

[1]	林威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6.
[2]	梁天恺, 黄康华, 刘凯航, 兰岚, 曾碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40.
[3]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[4]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[5]	马泽宇, 叶宁, 徐康, 王甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107.
[6]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[7]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[8]	沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9.
[9]	刘禅奕, 黄丹, 薛林雁, 王涛, 朱桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99.
[10]	马国祥, 杨凌菲, 严传波, 张志豪, 孙彬, 王晓荣. 基于深度DenseNet网络的肝包虫病超声影像诊断方法[J]. 计算机与现代化, 2023, 0(09): 100-104.
[11]	农皓程, 任德均, 任秋霖, 刘澎笠, 黄德成. 基于改进ConvNeXt的软塑包装表面异常检测算法[J]. 计算机与现代化, 2023, 0(08): 12-17.
[12]	欧阳飞, 吴旭, 向东升. 基于改进YOLOX的垃圾分类检测方法[J]. 计算机与现代化, 2023, 0(08): 68-73.
[13]	胡睿杰, 车逗. 红外小目标检测方法综述[J]. 计算机与现代化, 2023, 0(08): 79-86.
[14]	江蕾, 唐建, 杨超越, 吕婷婷. 基于CWGAN-GP与CNN的轴承故障诊断方法[J]. 计算机与现代化, 2023, 0(07): 1-6.
[15]	张志霞, 谢宝强. 基于FCGA-LSTM与迁移学习的天然气负荷预测[J]. 计算机与现代化, 2023, 0(07): 7-12.