Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation

doi:10.3969/j.issn.1006-2475.2024.01.012

Abstract

Abstract:

Abstract: An unsupervised domain adaptation for LiDAR semantic segmentation method is proposed to deal with the problem of excessive data required for semantic segmentation network training in outdoor large-scale scenes. The method uses a modified RandLA-Net for semantic segmentation using a small number of point clouds from the SPTLS3D’s real world data as target objects. The model finishes the pre-training of the segmentation network on SensatUrban， and completes the transfer task by minimizing the domain gap between the source and target domains. The RandLA-Net losses the global features of the original point cloud in the encoding process， so an additional method of obtaining global information to join the network decoding is proposed. In addition， for getting the differentiated information， the weights of the local attention module of RandLA-Net is changed to use the difference between the features of each point and the average features of its neighbors. The experiments show that the mean intersection over union of the network are 54.3% on SemanticKITTI and 71.91% on Semantic3D. The mIoU of the pre-trained network after fine-tuning are 80.05%， which is 8.83 percentage points better than training directly.

Key words: Key words: point cloud semantic segmentation, unsupervised domain adaptation, transfer learning, fine-tune, deep learning

CLC Number:

TP393

HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation[J]. Computer and Modernization, 2024, 0(01): 74-79.

References

［1］ BOULCH A， LE SAUX B， AUDEBERT N. Unstructured point cloud semantic labeling using deep segmentation networks［C］// Proceedings of the Workshop on 3D Object Retrieval. 2017:17-24.
［2］ LAWIN F J， DANELLJAN M， TOSTEBERG P， et al. Deep projective 3D semantic segmentation［C］// Proceedings of 17th International Conference on Computer Analysis of Images and Patterns. 2017:95-107.
［3］ KIM M， ILYAS N， KIM K. AMSASeg: An attention-based multi-scale atrous convolutional neural network for real-time object segmentation from 3D point cloud［J］. IEEE Access， 2021，9:70789-70796.
［4］ CHENG R， RAZANI R， TAGHAVI E， et al. （AF）2-S3Net: Attentive feature fusion with adaptive feature selection for sparse semantic segmentation network［C］. // Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:12542-12551.
［5］ ZHU X G， ZHOU H， WANG T， et al. Cylindrical and asymmetrical 3D convolution networks for LiDAR segmentation［C］// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:9934-9943.
［6］杨晓文，李静，韩燮，等. 基于八叉树的卷积神经网络三维模型分割［J］. 计算机工程与设计， 2020，41（9）:2663-2669.
［7］ QI C R， SU H， KAICHUN M， et al. PointNet: Deep learning on point sets for 3D classification and segmentation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:77-85.
［8］ QI C R， YI L， SU H， et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space［C］// Proceedings of the 31st Annual Conference on Neural Information Processing Systems. 2017:5105-5114.
［9］ THOMAS H， QI C R， DESCHAUD J E， et al. KPconv: Flexible and deformable convolution for point clouds［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 2019:6410-6419.
［10］ MA X， QIN C， YOU H X， et al. Rethinking network design and local geometry in point cloud: A simple residual MLP framework［J］. arXiv preprint arXiv:2202.07123， 2022.
［11］ XU J Y， ZHANG R X， DOU J， et al. RPVNet: A deep and efficient range-point-voxel fusion network for LiDAR point cloud segmentation［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. 2021:16004-16013.
［12］ ZHOU H， ZHU X G， SONG X， et al. Cylinder3D: An effective 3D framework for driving-scene lidar semantic segmentation ［J］. arXiv preprint arXiv:2008.01550， 2020.
［13］ LANDRIEU L， SIMONOVSKY M. Large-scale point cloud semantic segmentation with superpoint graphs［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:4558-4567.
［14］ HU Q Y， YANG B， XIE L H， et al. RandlA-Net: Efficient semantic segmentation of large-scale point clouds［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:11105-11114.
［15］ SHUAI H， XU X， LIU Q S. Backward attentive fusing network with local aggregation classifier for 3D point cloud semantic segmentation［J］. IEEE Transactions on Image Processing， 2021，30:4973-4984.
［16］苏鸣方，胡立坤，黄润辉. 基于上下文注意力的室外点云语义分割方法［J］. 计算机工程， 2023，49（3）:248-256.
［17］ CHEN M D， HU Q Y， YU Z F， et al. STPLS3D: A large-scale synthetic and real aerial photogrammetry 3D point cloud dataset［J］. arXiv preprint arXiv:2203.09065， 2022.
［18］ WU B C， ZHOU X Y， ZHAO S C， et al. SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud［C］// 2019 International Conference on Robotics and Automation. 2019:4376-4382.
［19］ XIAO A R， HUANG J X， GUAN D Y， et al. Transfer learning from synthetic to real LiDAR point cloud for semantic segmentation［C］. Proceedings of the AAAI Conference on Artificial Intelligence. 2022，36（3）:2795-2803.
［20］ HOFFMAN J， WANG D Q， YU F S， et al. FCNs in the wild: Pixel-level adversarial and constraint-based adaptation［J］. arXiv preprint arXiv:1612.02649， 2016.
［21］ WU B C， WAN A L， YUE X Y， et al. SqueezeSeg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud［C］// 2018 IEEE International Conference on Robotics and Automation. 2018:1887-1893.
［22］ ZHAO S C， WANG Y Z， LI B， et al. ePointDA: An end-to-end simulation-to-real domain adaptation framework for LiDAR point cloud segmentation［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2021，35（4）:3500-3509.
［23］ HU Q Y， YANG B， KHALID S， et al. Towards semantic segmentation of urban-scale 3D point clouds: A dataset， benchmarks and challenges ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:4975-4985.
［24］ RONNEBERGER O， FISCHER P， BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015:234-241.
［25］ LI Y Y， BU R， SUN M C， et al. PointCNN: Convolution on X-transformed points［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018:828-838.
［26］ SUN B C， SAENKO K. Deep coral: Correlation alignment for deep domain adaptation［C］// European Conference on Computer Vision. 2016:443-450.
［27］ HACKEL T， SAVINOV N， LADICKY L， et al. Semantic3D.net: A new large-scale point cloud classification benchmark［J］. arXiv preprint arXiv:1704.03847， 2017.
［28］ BEHLEY J， GARBADE M， MILIOTO A， et al. SemanticKITTI: A dataset for semantic scene understanding of lidar sequences［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 2019:9296-9306.

[1]	QI Xian, LIU Daming, CHANG Jiaxin. Multi-view 3D Reconstruction Based on Improved Self-attention Mechanism [J]. Computer and Modernization, 2024, 0(11): 106-112.
[2]	CHEN Kai1, LI Yiting1, 2, QUAN Huafeng1. A River Discarded Bottles Detection Method Based on Improved YOLOv8 [J]. Computer and Modernization, 2024, 0(11): 113-120.
[3]	YANG Jun1, HU Wei1, ZHU Wenfu2. Visual SLAM Loop Closure Detection Algorithm Based on Improved MobileNetV3 [J]. Computer and Modernization, 2024, 0(10): 21-26.
[4]	WANG Yingying, HAO Xiao. Fine-grained Image Classification Based on Res2Net and Recursive Gated Convolution [J]. Computer and Modernization, 2024, 0(10): 74-79.
[5]	SHI Xingyu1, LI Qiang2, ZHUANG Li3, LIANG Yi3, WANG Qiulin3, CHEN Kai3, WU Chenzhou3, CHANG Sheng1. Object Detection Models Distillation Technique for Industrial Deployment [J]. Computer and Modernization, 2024, 0(10): 93-99.
[6]	ZHANG Ze1, ZHANG Jianquan2, 3, ZHOU Guopeng2, 3. Camera Module Defect Detection Based on Improved YOLOv8s [J]. Computer and Modernization, 2024, 0(09): 107-113.
[7]	CHENG Yazi1, LEI Liang1, 2, CHEN Han1, ZHAO Yiran1. Multi-scale Depth Fusion Monocular Depth Estimation Based on Transposed Attention [J]. Computer and Modernization, 2024, 0(09): 121-126.
[8]	CHENG Meng, LI Hao. Improved Deciduous Tree Nest Detection Method Based on YOLOv5s [J]. Computer and Modernization, 2024, 0(08): 24-29.
[9]	WANG Mengxi, LI Jun. Review of Fall Detection Technologies for Elderly [J]. Computer and Modernization, 2024, 0(08): 30-36.
[10]	MA Yong, WANG Jun, ZHANG Zijian, ZHAO Yuyang, ZHANG Jing, ZHOU Ming. Improved YOLOv8 Behavior Detection Algorithm for Intelligent Operation and#br# Maintenance System [J]. Computer and Modernization, 2024, 0(08): 43-48.
[11]	SHI Xianwei1, FAN Xin2. Semantic Segmentation of Video Frame Scene Based on Lightweight [J]. Computer and Modernization, 2024, 0(08): 49-53.
[12]	XU Xin’ai, LI Gang. An Image Generation Method of Classroom Expression Images [J]. Computer and Modernization, 2024, 0(08): 88-91.
[13]	GAO Shuaipeng, WANG Yifan. Survey on Group-level Emotion Recognition in Images [J]. Computer and Modernization, 2024, 0(08): 98-107.
[14]	HUANG Wendong, WANG Yifan. Survey on Multimodal Information Processing and Fusion Based on Modal Categories [J]. Computer and Modernization, 2024, 0(07): 47-62.
[15]	WU Li1, ZHANG Zhenghao2, GE Caicheng2, YU Jun2. Lane Line Detection Algorithm Based on Improved SCNN Network [J]. Computer and Modernization, 2024, 0(07): 87-92.