Single View 3D Hair Modeling Based on Mask Constraints

doi:10.3969/j.issn.1006-2475.2025.09.013

Abstract

Abstract:
Abstract: A 3D hair reconstruction method based on Mask constraints is proposed. Firstly， the hair front view Mask is inferred using SAM， and a reasonable back view Mask is predicted by GAN network， the depth pseudo-labeling is introduced to fine-tune the Hourglass network to predict the hair depth map and enhance the relative positional relationship of the hair， and in addition， the U-Net is used to infer the hair orientation map， which is inputted into the Stacked Hourglass network， and respectively the front-view Mask and back-view Mask constraints to generate 3D hair spatial points to ensure that the generated 3D point cloud of hair is projected reasonably within the Mask range. Meanwhile， the robustness of the model at the hair edge is enhanced by controlling the negative sample sampling rate. Finally， a parallel algorithm is used to accelerate the hair synthesis and significantly improve the reconstruction efficiency. The experimental results show that the method is effective in dealing with complex hairstyles and improving the realism of hair.

Key words: Key words: hair modeling, image-based hair modeling, occupation field, 3D reconstruction, direction field

CLC Number:

中图分类号：TP391

LI Su, LI Dequan, QING Yu. Single View 3D Hair Modeling Based on Mask Constraints[J]. Computer and Modernization, 2025, 0(09): 90-96.

References

［1］ CAO X D， WEI Y C， WEN F， et al. Face alignment by explicit shape regression［J］. International Journal of Computer Vision， 2014，107:177-190.
［2］ HADAP S， CANI M P， LIN M， et al. Strands and hair: Modeling， animation， and rendering［M］// ACM SIGGRAPH
2007 Courses. ACM， 2007:1-150.
［3］ HU L W， SAITO S， WEI L Y， et al. Avatar digitization from a single image for real-time rendering［J］. ACM Transactions on Graphics （ToG）， 2017，36（6）:1-14.
［4］ LI H， TRUTOIU L， OLSZEWSKI K， et al. Facial performance sensing head-mounted display［J］. ACM Transactions on Graphics （ToG）， 2015，34（4）:1-9.
［5］ NAM G， WU C L， KIM M H， et al. Strand-accurate multi-view hair capture［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2019:155-164.
［6］ DEGRAEN D， ZENNER A， KRÜGER A. Enhancing texture perception in virtual reality using 3D-printed hair structures［C］// Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM， 2019: 1-12.
［7］ WOHLGENANNT I， SIMONS A， STIEGLITZ S. Virtual reality［J］. Business & Information Systems Engineering， 2020， 62（5）: 455-461.
［8］胡利军，庄科旻，杨豪，等. 基于虚拟现实技术的信息管理系统设计［J］. 计算机技术与发展， 2020，30（3）:157-161.
［9］王雅妮，高全力，王西汉，等. 虚拟环境下的低延迟协同渲染方法研究［J］. 计算机技术与发展， 2023，33（12）:121-127.
［10］ KIM S H. Development of the 3D hair style simulator using augmented reality［J］. Journal of Digital Convergence， 2015，13（1）:249-255.
［11］贾一鑫，邓魏永，殷强，等. 基于改进SURF的增强现实图像匹配方法［J］. 计算机技术与发展， 2024，34（1）:59-64.
［12］ HU L W， MA C Y， LUO L J， et al. Single-view hair modeling using a hairstyle database［J］. ACM Transactions on Graphics （ToG）， 2015，34（4）:1-9.
［13］ ZHANG M， WU P， WU H Z， et al. Modeling hair from an RGB-d camera［J］. ACM Transactions on Graphics （ToG）， 2018，37（6）:1-10.
［14］ CHAI M L， SHAO T J， WU H Z， et al. AutoHair: Fully automatic hair modeling from a single image［J］. ACM Transactions on Graphics（ToG）， 2016，35（4）:1-12.
［15］ ZHOU Y， HU L W， XING J， et al. HairNet: Single-view hair reconstruction using convolutional neural networks［C］// Proceedings of the 2018 European Conference on Computer Vision （ECCV）. Springer， 2018:249-265.
［16］ WU K Y， YE Y F， YANG L C， et al. NeuralHDHair: Automatic high-fidelity hair modeling from a single image using implicit neural representations［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2022:1526-1535.
［17］ ZHENG Y J， JIN Z R， LI M R， et al. HairStep: Transfer synthetic to real using strand and depth maps for single-view 3D hair modeling［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2023:12726-12735.
［18］ CHAI M L， WANG L D， WENG Y L， et al. Single-view hair modeling for portrait manipulation［J］. ACM Transactions on Graphics （ToG）， 2012，31（4）:1-8.
［19］ LUO L J， LI H， PARIS S， et al. Multi-view hair capture using orientation fields［C］// 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2012:1490-1497.
［20］ ZHANG M， ZHENG Y Y. Hair-GAN: Recovering 3D hair structure from a single image using generative adversarial networks［J］. Visual Informatics， 2019，3（2）:102-112.
［21］ MING K W， HIROKI T， MASAYUKI N. Generation of 3D hair model from 2D image using image processing［C］// Applications of Digital Image Processing XIX. SPIE， 1996， 2847:303-311.
［22］ SHEN Y F， ZHANG C G， FU H B， et al. DeepSketchHair: Deep sketch-based 3D hair modeling［J］. IEEE Transactions on Visualization and Computer Graphics， 2021，27（7）: 3250-3263.
［23］ KIRILLOV A， MINTUN E， RAVI N， et al. Segment anything［C］// Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. IEEE， 2023:3992-4003.
［24］ PARIS S， BRICENO H M， SILLION F X. Capture of hair geometry from multiple images［J］. ACM Transactions on Graphics （ToG）， 2004，23（3）:712-719.
［25］ YANG L C， SHI Z F， ZHENG Y Y， et al. Dynamic hair modeling from monocular videos using deep neural networks［J］. ACM Transactions on Graphics （ToG）， 2019，38（6）:1-12.
［26］ CHEN W F， FU Z， YANG D W， et al. Single-image depth perception in the wild［J］. Advances in Neural Information Processing Systems， 2016， 29.
［27］ RONNEBERGER O， FISCHER P， BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Springer， 2015:234-241.
［28］ ISOLA P， ZHU J Y， ZHOU T H， et al. Image-to-image translation with conditional adversarial networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2017:5967-5976.
［29］ YANG L H， KANG B Y， HUANG Z L， et al. Depth Anything V2［J］. arXiv preprint arXiv:2406.09414， 2024.
［30］ MIRZA M， OSINDERO S. Conditional generative adversarial nets［J］. arXiv preprint arXiv:1411.1784， 2014.

[1]	QI Xian, LIU Daming, CHANG Jiaxin. Multi-view 3D Reconstruction Based on Improved Self-attention Mechanism [J]. Computer and Modernization, 2024, 0(11): 106-112.
[2]	YE Senhui, WANG Lei. Multi-view Reconstruction with Local Self-attention and Deep Optimization [J]. Computer and Modernization, 2024, 0(05): 92-98.
[3]	ZHANG Ya-wen, LIN Wen-zhong, HAN Xiao-dong. Point Cloud Registration Algorithm Based on Combined Feature Points and#br# Principal Component Analysis#br# [J]. Computer and Modernization, 2023, 0(09): 59-63.
[4]	ZHANG Wen-yuan, HAI Lin-qi, LIU Ying-jie, ZHANG Hai-bo. An Improved Multispectral Reconstruction for Sparse-view CB-XLCT Imaging [J]. Computer and Modernization, 2022, 0(05): 96-101.
[5]	CHEN Kai-yang1, LUO Zhi-zao2, WANG Jian-xing2. Improvement for Feature Point Extraction Based on Kinect 3D Reconstruction [J]. Computer and Modernization, 2019, 0(11): 34-.
[6]	LOU Xingliang, ZHANG Weiping, CAI Xuefei, ZOU Yang. 3D Reconstruction of FMAV Based on MATLAB [J]. Computer and Modernization, 2017, 0(8): 74-.
[7]	HUA Chunjian1,2， FANG Chengjun1,2， CHEN Ying3. A Freeform Surface Stereo Matching Method Based on Improved Census Transform [J]. Computer and Modernization, 2015, 0(7): 44-.
[8]	WEI Yu-mian, SHANG Zhao-wei. 3D Reconstruction Method of Rotational Rigid Object Based on Kinect [J]. Computer and Modernization, 2014, 0(5): 89-93，98.
[9]	DONG Xiao-long;GUO Tai-liang. Three-dimensional Reconstruction Method Based on Decorrelation of Speckle Patterns [J]. Computer and Modernization, 2012, 203(7): 83-87.
[10]	DANG Guang-lei;WEI Wei. Design of Medical Image System Based on MITK [J]. Computer and Modernization, 2011, 1(2): 8-10.
[11]	SHI Jin-long;LIU Qing-hua. A 3D Reconstruction Method for Deformable Surface [J]. Computer and Modernization, 2011, 1(11): 64-3.