Computer and Modernization ›› 2024, Vol. 0 ›› Issue (05): 92-98.doi: 10.3969/j.issn.1006-2475.2024.05.016

Previous Articles     Next Articles

Multi-view Reconstruction with Local Self-attention and Deep Optimization

  



  1. (School of Information Engineering, East China University of Technology, Nanchang 330013, China)
  • Online:2024-05-29 Published:2024-06-12

Abstract: Abstract: To address the issues of high memory and time consumption, low completeness and fidelity of high-resolution reconstruction in multi-view 3D reconstruction, we propose a deep learning-based multi-view reconstruction network. The network consists of a feature extraction module, a cascaded Patchmatch module and a depth map optimization module. First, we design a U-shaped feature extraction module to extract multi-stage feature maps, and introduce local self-attention layers with relative position encoding at each stage, which capture the local details and global context in the images, and enhance the feature extraction performance of the network. Second, we design a deep residual network to fuse the features, and fully utilize the color image prior knowledge to constrain the depth map, and improve the accuracy of depth estimation. We test our network on the public dataset DTU (Technical University of Denmark), and the experimental results show that our network achieves significant improvement in 3D reconstruction quality. Compared with PatchmatchNet, our network improves the completeness by 6.1% and the overall by 2.5%. Compared with other SOTA (State-Of-The-Art) methods, our network also achieves better performance in both completeness and overall.

Key words: Key words: deep learning, 3D reconstruction, local self-attention mechanism, multi-view stereo, depth estimation

CLC Number: