Point Cloud Completion Algorithm Based on Multi-stage Fractal Combination

doi:10.3969/j.issn.1006-2475.2023.12.005

Abstract

Abstract: Abstract: Point cloud is a common representation of 3D objects. However， due to reasons such as sensor design and precision， the obtained point cloud usually has missing geometry and sparseness. To solve this problem， this paper proposes a point cloud completion algorithm based on multi-stage fractal combination. In the first stage， the input point cloud is sampled multiple times and features are extracted separately， then the pyramid model is used to generate a point cloud with multi-scale geometry loss， and finally， the generated point cloud is spliced with the input point cloud. In the second stage， KNN clustering and PointNet stacking network are used to extract local features， and the spliced point cloud is down-sampled as a rough prediction， and finally， the rough prediction is combined with the local input folding network to generate a refined high-quality point cloud. This algorithm is based on local to overall multi-stage completion， and the loss function can be adjusted for different stages， which effectively optimizes the completion process and achieves good completion results in the ShapeNet dataset.

Key words: Key words: point cloud completion, multi-stage, fractal combination, missing geometry, missing sparsity

CLC Number:

TP391

ZENG Wei-ping, CHEN Jun-hong, Muhammad ASIM, LIU Wen-yin, YANG Zhen-guo. Point Cloud Completion Algorithm Based on Multi-stage Fractal Combination[J]. Computer and Modernization, 2023, 0(12): 24-29.

References

［1］ ZENG Y M， HU Y， LIU S C， et al. RT3D: Real-time 3-D vehicle detection in LiDAR point cloud for autonomous driving［J］. IEEE Robotics and Automation Letters， 2018，3（4）:3434-3440.
［2］ RAHMAN M M， TAN Y H， XUE J， et al. Recent advances in 3D object detection in the era of deep neural networks: A survey［J］. IEEE Transactions on Image Processing， 2020，29:2947-2962.
［3］ HU Q Y， YANG B， XIE L H， et al. RandLA-Net: Efficient semantic segmentation of large-scale point clouds［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2020:11105-11114.
［4］ HAN X F， LAGA H， BENNAMOUN M. Image-based 3D object reconstruction: State-of-the-art and trends in the deep learning era［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021，43（5）:1578-1604.
［5］ HU W， FU Z Q， GUO Z M. Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting［J］. IEEE Transactions on Image Processing， 2019，28（8）:4087-4100.
［6］ MARTINOVIC A， GOOL L V. Bayesian grammar learning for inverse procedural modeling［C］// 2013 IEEE Conference on Computer Vision and Pattern Recognition. 2013:201-208.
［7］ YUAN W T， KHOT T， HELD D， et al. PCN: Point completion network［C］// 2018 International Conference on 3D Vision （3DV）. 2018:728-737.
［8］ HUANG Z T， YU Y K， XU J W， et al. PF-Net: Point fractal network for 3D point cloud completion［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2020:7659-7667.
［9］ YU X M， RAO Y M， WANG Z Y， et al. PoinTr: Diverse point cloud completion with geometry-aware transformers.［C］// 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021: 12478-12487.
［10］ YI L， KIM V G， CEYLAN D， et al. A scalable active framework for region annotation in 3D shape collections［J］. ACM Transactions on Graphics， 2016，35（6）. DOI:10.1145/2980179.2980238
［11］ HAN Z Z， LIU Z B， HAN J W， et al. Mesh convolutional restricted Boltzmann machines for unsupervised learning of features with structure preservation on 3-D meshes［J］. IEEE Transactions on Neural Networks and Learning Systems， 2017，28（10）:2268-2281.
［12］ LIU X H， HAN Z Z， LIU Y S， et al. Point2Sequence: Learning the shape representation of 3D point clouds with an attention-based sequence to sequence network［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2019，33（1）:8778-8785.
［13］ HAN Z Z， LIU Z B， VONG C M， et al. BoSCC: Bag of spatial context correlations for spatially enhanced 3D shape representation［J］. IEEE Transactions on Image Processing， 2017，26（8）: 3707-3720.
［14］ HAN Z Z， LIU X H， LIU Y S， et al. Parts4Feature: Learning 3D global features from generally semantic parts in multiple views［J］. arXiv preprint arXiv:1905.07506， 2019.
［15］ WANG Y， SOLOMON J. Deep closest point: Learning representations for point cloud registration［C］// 2019 IEEE/CVF International Conference on Computer Vision （ICCV）. 2019:3522-3531.
［16］ HAN Z Z， SHANG M Y， LIU Y S， et al. View Inter-prediction GAN: Unsupervised representation learning for 3D shapes by learning global shape memories to support local view predictions［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2019，33（1）:8376-8384.
［17］ HAN Z Z， LIU H L， LIU Z B， et al. 3D2SeqViews: Aggregating sequential views for 3D global feature learning by CNN with hierarchical attention aggregation［J］. IEEE Transactions on Image Processing， 2019，28（8）:3986-3999.
［18］ LAWIN F J， DANELLJAN M， TOSTEBERG P， et al. Deep projective 3D semantic segmentation［C］// International Conference on Computer Analysis of Images and Patterns. 2017:95-107.
［19］BEN-SHABAT Y， LINDENBAUM M， FISCHER A. 3DmFV: Three-dimensional point cloud classification in real-time using convolutional neural networks.［J］. IEEE Robotics and Automation Letters， 2018，3（4）:3145-3152.
［20］ CHARLES R Q， SU H， KAICHUN M， et al. PointNet: Deep learning on point sets for 3D classification and segmentation［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017:77-85.
［21］ QI C R， YI L， SU H， et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:5105-5114.
［22］ YANG Y Q， FENG C， SHEN Y R， et al. FoldingNet: Point cloud auto-encoder via deep grid deformation［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:206-215.
［23］ ZHANG J Z， CHEN X Y， CAI Z G， et al. Unsupervised 3D shape completion through GAN inversion［C］// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2021:1768-1777.
［24］ BARSOUM E， KENDER JOHN， LIU Z C. HP-GAN: Probabilistic 3D human motion prediction via GAN［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 2018:1499-1508.
［25］ ZHANG X C， FENG Y T， LI S Q， et al. View-guided point cloud completion［C］// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2021:15885-15894.
［26］ WOLF T， DEBUT L， SANH V， et al. Transformers: State-of-the-art natural language processing［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2020:38-45.
［27］ WANG Y， SUN Y B， LIU Z W， et al. Dynamic graph CNN for learning on point clouds［J］. ACM Transactions on Graphic， 2019，38（5）. DOI: 10.1145/3326362.
［28］ FAN H Q， SU H， GUIBAS L J. A point set generation network for 3D object reconstruction from a single image［J］. arXiv preprint arXiv:1612.00603v2， 2016.

[1]	MENG Xi-ting, JI Lu-yan, . A Multi-stage Remote Sensing Image Object Detection Method [J]. Computer and Modernization, 2020, 0(09): 66-72.
[2]	WANG Fang;WANG Wei. Load Schedule Strategy of Streaming Media Cluster Server Based on Response Times Estimation [J]. Computer and Modernization, 2013, 1(5): 108-111,.