多视图IM-NET三维目标精细化重建

doi:10.3969/j.issn.1006-2475.2025.12.007

计算机与现代化 ›› 2025, Vol. 0 ›› Issue (12): 46-53.doi: 10.3969/j.issn.1006-2475.2025.12.007

多视图IM-NET三维目标精细化重建

(1.赣南师范大学科技学院，江西赣州 341000； 2.广东茂名幼儿师范专科学校计算机学院，广东茂名 525000；
3.广州华商学院，广东广州 511300； 4.江西理工大学信息工程学院，江西赣州 341000)

出版日期:2025-12-18 发布日期:2025-12-18
作者简介:作者简介：刘健龙(2000—),男,江西赣州人,硕士,研究方向:图像三维重建,E-mail: liujianlong00@hotmail.com; 岑颖(1981—),男,广东高州人,助教,硕士,研究方向:计算机视觉,E-mail: 13553683306@139.com; 许斌(2000—),男,江西吉安人,硕士,研究方向:计算机视觉,E-mail: Xb_200010@163.com; 焦旋(2001—),男,河南新乡人,硕士研究生,研究方向:图像三维重建,E-mail: 6720230834@mail.jxust.edu.cn。
基金资助:
基金项目：江西省自然科学基金资助项目(20224BAB202002)

Multi-view IM-NET for Fine Reconstruction of 3D Object

(1. College of Science and Technology, Gannan Normal University, Ganzhou 341000, China; 2. School of Computing, Guangdong
Preschool Normal College in Maoming, Maoming 525000, China; 3. Guangzhou Huashang College, Guangzhou 511300, China;
4. School of Information Engineering, Jiangxi University of Science and Technology, Ganzhou 341000, China)

Online:2025-12-18 Published:2025-12-18

摘要/Abstract

摘要： 摘要：隐式表达网络(Implicit Net, IM-NET)将三维目标重建转换为空间中的采样点是否在目标表面或内部的分类问题，有效地节约了计算和存储资源。然而，IM-NET只能从单幅视图中重建三维目标，单视图中有限的目标信息量尤其是被遮挡部分的信息缺失，导致了目标重建精度的不足。本文将单视图IM-NET推广至多视图输入，采用注意力模块对多视图中提取出的特征进行融合，以获取更为完整的目标特征，从而提升三维重建精度。考虑到从隐式表达到网格显示表达的转换过程中，Marching Cubes算法所生成的目标表面不够平滑，本文进一步采用网格细化方法对重建目标进行多次迭代细化，以实现精细化重建。在ShapNet数据集上的实验结果表明，与单视图IM-NET和其他多视图重建方法相比，本文提出的多视图IM-NET重建出的三维目标更为完整和平滑，且目标的平均交并比得到大幅度提升。此外，可视化效果表明，经过细化后的目标细节更丰富且表面更平滑。

关键词: 关键词：深度学习, 三维重建, IM-NET, 多视图

Abstract: Abstract: Implicit net (IM-NET) converts the task of reconstructing three-dimensional (3D) objects into a classification issue concerning whether sampled points in space are on or inside the object’s surface, effectively conserving computational and storage resources. However, IM-NET can only reconstruct 3D objects from a single view, and the limited amount of target information available in a single view, especially the missing information from occluded parts, leads to insufficient reconstruction accuracy. This paper extends IM-NET to adapt multi-view inputs by employing an attention module to fuse extracted features, thereby obtaining more complete target features and enhancing 3D reconstruction accuracy. Considering that the target surface generated by the Marching Cubes algorithm is not smooth enough when converting implicit expression into explicit mesh representation, this paper utilizes a mesh refinement algorithm to refine the reconstructed targets iteratively, achieving the refined reconstruction. Experimental results on the ShapNet dataset indicate that compared with single-view IM-NET and other multi-view reconstruction methods, the proposed multi-view IM-NET reconstructs 3D targets more complete and smooth, and the average intersection and union ratios of targets are significantly improved. Additionally, visualization effects show that the refined targets have richer details and smoother surfaces than those without refinement.

Key words: Key words: deep learning, 3D reconstruction, IM-NET, multi-view

中图分类号:

中图分类号:TP391

刘健龙1, 岑颖2, 许斌3, 焦旋4. 多视图IM-NET三维目标精细化重建[J]. 计算机与现代化, 2025, 0(12): 46-53.

LIU Jianlong1, CEN Ying2, XU Bin3, JIAO Xuan4. Multi-view IM-NET for Fine Reconstruction of 3D Object[J]. Computer and Modernization, 2025, 0(12): 46-53.

参考文献

[1] RICHTER S R, ALHAIJA H A, KOLTUN V. Enhancing photorealism enhancement[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023,45(2):1700-1715.
[2] XIANG L R, GAI J Y, BAO Y, et al. Field-based robotic leaf angle detection and characterization of maize plants using stereo vision and deep convolutional neural networks[J]. Journal of Field Robotics, 2023,40(5):1034-1053.
[3] DING C L, DAI Y F, FENG X M, et al. Stereo vision SLAM-based 3D reconstruction on UAV development platforms[J]. Journal of Electronic Imaging, 2023,32(1). DOI:
10.1117/1.JEI.32.013041.
[4] SUMETHEEPRASIT B, ROSALES MARTINEZ R, PAUL H, et al. Variable baseline and flexible configuration stereo vision using two aerial robots[J]. Sensors, 2023,23(3). DOI: 10.3390/S23031134.
[5] PETRAKIS G, ANTONOPOULOS A, TRIPOLITSIOTIS A, et al. Precision mapping through the stereo vision and geometric transformations in unknown environments[J]. Earth Science Informatics, 2023,16(2):1849-1865.
[6] 杨航,陈瑞,安仕鹏,等. 深度学习背景下的图像三维重建技术进展综述[J]. 中国图象图形学报, 2023,28(8):2396-2409.
[7] CHOY C B, XU D F, GWAK J Y, et al. 3D-R2N2: A unified approach for single and multi-view 3D object reconstruction[C]// Proceedings of the 14th European Conference on Computer Vision(ECCV 2016). Springer, 2016:628-644.
[8] TATARCHENKO M, DOSOVITSKIY A, BROX T. Octree generating networks:Efficient convolutional architectures for high-resolution 3D outputs[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. IEEE, 2017:2088-2096.
[9] FAN H Q, SU H, GUIBAS L J. A point set generation network for 3D object reconstruction from a single image[C]// 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2017:605-613.
[10] MELAS-KYRIAZI L, RUPPRECHT C, VEDALDI A. PC2:Projection-conditioned point cloud diffusion for single-image 3D reconstruction[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2023:12923-12932.
[11] GROUEIX T, FISHER M, KIM V G, et al. A papier-mâché approach to learning 3D surface generation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2018:216-224.
[12] WANG N Y, ZHANG Y D, LI Z W, et al. Pixel2Mesh: Generating 3D mesh models from single RGB images[C]// Proceedings of the 2018 European Conference on Computer Vision(ECCV). Springer, 2018:55-71.
[13] WEN C, ZHANG Y D, LI Z W, et al. Pixel2Mesh++: Multi-view 3D mesh generation via deformation[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. IEEE, 2019:1042-1051.
[14] LIN K, WANG L J, LIU Z C. End-to-end human pose and mesh reconstruction with Transformers[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2021:1954-1963.
[15] CHEN Z Q, ZHANG H. Learning implicit fields for generative shape modeling[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2019:5939-5948.
[16] MESCHEDER L, OECHSLE M, NIEMEYER M, et al. Occupancy networks: Learning 3D reconstruction in function space[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2019:4460-4470.
[17] XU Q G, WANG W Y, CEYLAN D, et al. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction[J]. arXiv preprint arXiv:1905.10711, 2019.
[18] MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: Representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1):99-106.
[19] YANG Z Y, GAO X Y, ZHOU W, et al. Deformable 3D gaussians for high-fidelity monocular dynamic scene reconstruction[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2024:20331-20341.
[20] SAITO S, HUANG Z, NATSUME R, et al. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. IEEE, 2019:2304-2314.
[21] SAITO S, SIMON T, SARAGIH J, et al. PIFuHD: Multi-level pixel-aligned implicit function for high-resolution 3D human digitization[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2020:84-93.
[22] PENG S Y, NIEMEYER M, MESCHEDER L, et al. Convolutional occupancy networks[C]// Proceedings of the 2020 European Conference on Computer Vision(ECCV). Springer, 2020:523-540.
[23] TANG J P, LEI J B, XU D, et al. SA-ConvONet: Sign-agnostic optimization of convolutional occupancy networks[C]// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. IEEE, 2021:6504-6513.
[24] BOULCH A, MARLET R. POCO: Point convolution for surface reconstruction[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2022:6302-6314.
[25] LORENSEN W E, CLINE H E. Marching Cubes: A high resolution 3D surface construction algorithm[J]. ACM SIGGRAPH Computer Graphics, 1987,21(4):163-169.
[26] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). IEEE, 2016:770-778.
[27] YANG B, WANG S, MARKHAM A, et al. Robust attentional aggregation of deep feature sets for multi-view 3D reconstruction[J]. International Journal of Computer Vision, 2020,128(1):53-73.
[28] CHANG A X, FUNKHOUSER T, GUIBAS L, et al. ShapeNet:An information-rich 3D model repository[J]. arXiv preprint arXiv:1512.03012, 2015.

多视图IM-NET三维目标精细化重建

Multi-view IM-NET for Fine Reconstruction of 3D Object

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	汤辉1, 刘鑫2, 胡必伟1, 刘帆1. KD-NeRF：知识蒸馏增强的果树神经辐射场重建方法[J]. 计算机与现代化, 2025, 0(12): 66-73.
[2]	李雪, 魏延, 李林骏. 基于DWT-SCINet-MDSC的电价预测混合模型[J]. 计算机与现代化, 2025, 0(10): 37-43.
[3]	姚敏佳1, 宋文爱1, 孙雪2, 王子钰3, 雷毅4, 王青4, 赵莉5. InstantMesh：早期胃癌图像三维重建方法[J]. 计算机与现代化, 2025, 0(10): 51-56.
[4]	张婧颖1, 耿琳2, 刘宁钟2. 基于自蒸馏和自注意力增强的低数据细粒度图像分类[J]. 计算机与现代化, 2025, 0(09): 27-34.
[5]	李谡, 李德权, 庆宇. 基于Mask约束的单视图3D头发建模[J]. 计算机与现代化, 2025, 0(09): 90-96.
[6]	祁贤, 刘大铭, 常佳鑫. 基于改进自注意力机制的多视图三维重建[J]. 计算机与现代化, 2024, 0(11): 106-112.
[7]	叶森辉, 王蕾. 结合局部自注意力和深度优化的多视图重建[J]. 计算机与现代化, 2024, 0(05): 92-98.
[8]	游嘉靖1, 2, 何月顺1, 何璘琳1, 钟海龙1, 2. 基于AHP-CNN的加密流量分类方法[J]. 计算机与现代化, 2024, 0(04): 83-87.
[9]	陈震1, 姚京辉2, 苏成悦1. 基于改进U-Net的髋关节关键点检测算法[J]. 计算机与现代化, 2024, 0(02): 15-19.
[10]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[11]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[12]	张亚文, 林文忠, 韩晓东. 基于组合特征点和主成分分析的点云配准算法[J]. 计算机与现代化, 2023, 0(09): 59-63.
[13]	张文元, 海琳琦, 刘英杰, 张海波. 一种基于改进多光谱的稀疏角CB-XLCT成像方法[J]. 计算机与现代化, 2022, 0(05): 96-101.
[14]	苏桃, 邵盈鑫, 李纯斌, 吴静, 常秀红, 杨帆. 基于运动恢复结构方法的植株三维重建[J]. 计算机与现代化, 2020, 0(11): 94-99.
[15]	孟兆旭，蔡超. 环境重建三维格网数据下的快速航迹规划方法[J]. 计算机与现代化, 2017, 0(11): 17-22+34.