改进的神经渲染方法在建筑施工场景中的应用

doi:10.3969/j.issn.1006-2475.2023.12.013

摘要/Abstract

摘要： 摘要：针对神经辐射场（NeRF）应用于室外施工场景时，由于室外场景中光照难以捕捉和施工场景的前景背景范围差异过大，在新颖视图中会出现模糊和伪影等现象，通过分析提出改进的视觉合成方法。首先通过SFM算法从RGB图像中获得相机参数，实现对室外施工场景的表示；接着引入预训练编码器中生成的向量，并加入渲染网络中减少光照的影响；最终将图像中的前景与背景分离开进行体绘制渲染，从而提高视觉合成的效果。基于室外施工场景数据集，与3种方法进行比较，结果表明，所提方法在峰值信噪比（PSNR）和结构相似性（SSIM）中分别较其中最优方法提高12.2%与10.9%。整体看来，所提方法在室外施工场景中生成的新颖视图具有更好的细腻度。

关键词: 关键词：神经辐射场, 视觉合成, 施工场景, 光照, 范围

Abstract: Abstract: When neural radiation field （NeRF） is applied to outdoor construction scenes， due to the difficulty in capturing the lighting in outdoor scenes and the large difference between the foreground and background ranges of construction scenes， blurring and artifacts will appear in novel views. Through analysis， an improved visual synthesis method is proposed. First， camera parameters are obtained from RGB images through SFM algorithm to represent outdoor construction scenes. Then， the vector generated from the pre training encoder is introduced and added to the rendering network to reduce the impact of light. Finally， the foreground and background in the image are separated for volume rendering， which improves the effect of visual synthesis. Based on the outdoor construction scene data set， compared with three methods， the results show that the proposed method improves the peak signal-to-noise ratio and structural similarity by 12.2% and 10.9% respectively compared with the best method. It appears that the novel views generated by the proposed method in outdoor construction scenes have better finesse.

Key words: Key words: NeRF, visual synthesis, construction scenario, illumination, range

中图分类号:

TP391

张在成, 李健. 改进的神经渲染方法在建筑施工场景中的应用[J]. 计算机与现代化, 2023, 0(12): 76-81.

ZHANG Zai-cheng, LI Jian. Application of Neural Rendering Based Visual Synthesis in Construction Scene[J]. Computer and Modernization, 2023, 0(12): 76-81.

参考文献

［1］ TEWARI A， FRIED O， THIES J， et al. State of the art on neural rendering［J］. Computer Graphics Forum， 2020，39（2）:701-727.
［2］ SHUM H Y， HE L W. Rendering with concentric mosaics［C］// Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. 1999:299-306.
［3］ NIEMEYER M， MESCHEDER L， OECHSLE M， et al. Differentiable volumetric rendering: Learning implicit 3D representations without 3D supervision［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:3501-3512.
［4］ SITZMANN V， ZOLLHOFER M， WETZSTEIN G. Scene representation networks: Continuous 3D-structure-aware neural scene representations［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019:1119-1130.
［5］ MILDENHALL B， SRINIVASAN P P， TANCIK M， et al. NeRF: Representing scenes as neural radiance fields for view synthesis［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020:405-421.
［6］常远，盖孟. 基于神经辐射场的视点合成算法综述［J］. 图学学报， 2021，42（3）:376-384.
［7］ MARTIN-BRUALLA R， RADWAN N， SAJJADI M S M， et al. NeRF in the wild: Neural radiance fields for unconstrained photo collections［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:7206-7215.
［8］ ZHANG K， RIEGLER G， SNAVELY N， et al. NeRF++: Analyzing and improving neural radiance fields［J］. arXiv preprint arXiv:2010.07492， 2020.
［9］ DAVIS A， LEVOY M， DURAND F. Unstructured light fields［J］. Computer Graphics Forum， 2012，31（2pt1）:305-314.
［10］ BROCK A， LIM T， RITCHIE J M， et al. Generative and discriminative voxel modeling with convolutional neural networks［J］. arXiv preprint arXiv:1608.04236， 2016.
［11］ ACHLIOPTAS P， DIAMANTI O， MITLIAGKAS I， et al. Learning representations and generative models for 3D point clouds［C］// Proceedings of the 35th International Conference on Machine Learning. 2018:40-49.
［12］ GROUEIX T， FISHER M， KIM V G， et al. AtlasNet: A papier-mache approach to learning 3D surface generation［J］. arXiv preprint arXiv:1802.05384， 2018.
［13］ MICHALKIEWICZ M， PONTES J K， JACK D， et al. Implicit surface representations as layers in neural networks［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. 2019:4743-4752.
［14］ HARTLEY R， ZISSERMAN A. Multiple View Geometry in Computer Vision［M］. Cambridge University Press， 2003.
［15］ HIEP V H， KERIVEN R， LABATUT P， et al. Towards high-resolution large-scale multi-view stereo［C］// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. 2009:1430-1437.
［16］ HEDMAN P， PHILIP J， PRICE T， et al. Deep blending for free-viewpoint image-based rendering［J］. ACM Transactions on Graphics （TOG）， 2018，37（6）. DOI: 10.1145/3272127.3275084.
［17］ SCHONBERGER J L， FRAHM J M. Structure-from-motion revisited［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:4104-4113.
［18］ WAECHTER M， MOEHRLE N， GOESELE M. Let there be color! Large-scale texturing of 3D reconstructions［C］// Proceedings of the 2014 European Conference on Computer Vision. 2014:836-850.
［19］ CHEN W Z， LING H， GAO J， et al. Learning to predict 3D objects with an interpolation-based differentiable renderer［C］// Proceedings of the 2019 International Conference on Neural Information Processing Systems. 2019.
［20］ LI T M， AITTALA M， DURAND F， et al. Differentiable Monte Carlo ray tracing through edge sampling［J］. ACM Transactions on Graphics （TOG）， 2018，37（6）. DOI: 10.1145/3272127.3275109.
［21］ KUTULAKOS K N， SEITZ S M. A theory of shape by space carving［J］. International Journal of Computer Vision， 2000，38（3）:199-218.
［22］ MILDENHALL B， SRINIVASAN P P， ORTIZ-CAYON R， et al. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines［J］. ACM Transactions on Graphics （TOG）， 2019，38（4）. DOI: 10.1145/3306346.3322980.
［23］ WOOD D N， AZUMA D I， ALDINGER K， et al. Surface light fields for 3D photography［C］// Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. 2000:287-296.
［24］ ZHU J Y， ZHANG R， PATHAK D， et al. Toward multimodal image-to-image translation［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:465-476.
［25］ MCREYNOLDS T， BLYTHE D. Advanced Graphics Programming Techniques Using OpenGL［M］. Silicon Graphics， 1998.
［26］ ATTAL B， LING S， GOKASLAN A， et al. MatryODShka: Real-time 6DoF video view synthesis using multi-sphere images［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020:441-459.

[1]	吴忠得, 罗晓芳, 侯增选, 段鹏轩, 李楠楠. 基于三维地球场景的干扰条件雷达探测范围可视化方法[J]. 计算机与现代化, 2020, 0(10): 90-96.
[2]	桂易琪，田星晨. 一种结合专家信任的POI推荐算法[J]. 计算机与现代化, 2019, 0(12): 10-.
[3]	梁珺秀,许建秋. 基于时空标签轨迹的范围模式匹配查询[J]. 计算机与现代化, 2018, 0(08): 79-.
[4]	任菲,王家润,李小娟,尹辉. 基于OSG的三维雷达探测范围构建与优化实现[J]. 计算机与现代化, 2018, 0(04): 111-.
[5]	罗光明. 提高人脸识别率的重光照算法[J]. 计算机与现代化, 2015, 0(6): 51-54.
[6]	王华君，李荣，徐燕华，孟德建. 基于局部相位纹理表示的光照变化人脸识别算法[J]. 计算机与现代化, 2015, 0(12): 84-.
[7]	田配云;郑碧波;陈伟清. 基于彩色局部角度二值模式的纹理分类研究[J]. 计算机与现代化, 2013, 1(4): 86-89,9.
[8]	李楠. 一种光照变化条件下的人脸识别方法[J]. 计算机与现代化, 2012, 203(7): 104-106,.
[9]	潘浩;杨宜民. 一种基于光照变化补偿的颜色识别方法[J]. 计算机与现代化, 2012, 1(200): 201-04.
[10]	冯皓;滕忠坚;林杨;庄小芳. 位图图像稳定区域分割的种子点选取条件[J]. 计算机与现代化, 2012, 1(03): 44-47.
[11]	仇开金. 基于单幅图像明暗恢复形状算法设计[J]. 计算机与现代化, 2010, 1(6): 58-0060.