基于生成对抗网络的图像动漫化

摘要/Abstract

摘要： 动漫风格的图像具有高度的简化和抽象等特征，为了解决将现实世界图像转化成动漫风格图像这一问题，提出一种基于生成对抗网络的图像动漫化方法。本文的生成网络是类U-Net的全卷积结构，对输入图像先下采样，并加上浅层的特征用双线性插值的方法进行上采样，判别网络则采用Patch GAN加谱归一化的结构，分别计算语义内容损失和风格损失以提高网络的稳定性。本文采用surface表征损失、structure表征损失和texture表征损失代替风格损失，使得生成动漫图像的效果更可控。写实图像选用train2014，人脸图像采用CelebA-HQ数据集。使用本文模型在这些数据集上进行实验，实验结果表明，本文模型能够有效地完成图像动漫化的过程，并生成较高质量的动漫化图像。

关键词: 深度学习, 生成对抗网络, 图像动漫化

Abstract: Anime-style images are highly simplified and abstract. In order to solve the problem of transforming real-world images into anime-style images, this paper proposes an image animation method based on generative adversarial networks. The generation network in this paper is like a U-Net fully convolutional structure. The input image is down-sampled first, and the shallow features are up-sampled by bilinear interpolation. The discriminant network uses Patch GAN and spectrum normalization. Semantic content loss and style loss are calculated separately to improve the stability of the network. Surface representation loss, structure representation loss, and texture representation loss are used to replace style loss to make the effect of generating animation pictures more controllable. We use train2014 for realistic images, and use the CelebA-HQ data set for face images. Experiments are performed on these data sets using this model. The experimental results show that the model in this paper can effectively complete the process of image animation and generate high-quality animation images.

Key words: deep learning, generative adversarial networks, image animation

翟慧聪, 张明, 邓星, 王利群. 基于生成对抗网络的图像动漫化[J]. 计算机与现代化, 2022, 0(07): 21-26.

ZHAI Hui-cong, ZHANG Ming, DENG Xing, WANG Li-qun. Image Animation Based on Generative Adversarial Networks[J]. Computer and Modernization, 2022, 0(07): 21-26.

参考文献［27］

［1］	ROSIN P, COLLOMOSSE J. Image and Video-Based Art-istic Stylisation［M］. Springer, 2012.
［2］	GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style［J］. arXiv preprint arXiv:1508.06576, 2015.
［3］	JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution［J］. arXiv preprint arXiv:1603.08155, 2016.
［4］	GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:2414-2423.
［5］	MIRZA M, OSINDERO S. Conditional generative adversarial nets［J］. arXiv preprint arXiv:1411.1784, 2014.
［6］	ISOLA P, ZHU J Y, ZHOU T H, et al. Image-to-image translation with conditional adversarial networks［J］. arXiv preprint arXiv:1611.07004, 2018.
［7］	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［J］. arXiv preprint arXiv:1703.10593, 2018.
［8］	CHEN Y, LAI Y K, LIU Y J. CartoonGAN: Generative adversarial networks for photo cartoonization［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:9465-9474.
［9］	WANG X R, YU J Z. Learning to cartoonize using white-box cartoon representations［C］// Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. 2020:1120-1130.
［10］	LEE H Y, TSENG H Y, HUANG J B, et al. Diverse image-to-image translation via disentangled representations［C］// Proceedings of the 2018 European Conference on Computer Vision. 2018:35-51．
［11］	BIAN Y A, LI X, LIU Y C, et al. Parallel coordinate descent Newton method for efficient L1-regularized loss minimization［J］. IEEE Transactions on Neural Networks and Learning Systems, 2019,30(11):3233-3245．
［12］	ZHANG R, ISOLA P, EFROS A A. Colorful image colorization［C］// 2016 European Conference on Computer Vision. 2016:649-666.
［13］	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014,2:2672-2680.
［14］	SIMONYAN K, ZISSERMAN A. Very deep convolutional network for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556, 2014.
［15］	MAO X D, LI Q, XIE H R, et al. Least squares generative adversarial networks［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. 2017:2813-2821.
［16］	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Intervention. 2015:234-241.
［17］	YU J H, LIN Z, YANG J M, et al. Free-form image inpainting with gated convolution［C］// Proceedings of the 2019 IEEE /CVF International Conference on Computer Vision. 2019:4470-4479.
［18］	TONG T, LI G, LIU X J, et al. Image super-resolution using dense skip connections［C］// Proceedings of 2017 IEEE International Conference on Computer Vision. 2017:4809-4817.
［19］	LAN Z Z, LIN M, LI X C, et al. Beyond Gaussian pyramid: Multi-skip feature stacking for action recognition［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:204-212.
［20］	MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks［J］. arXiv preprint arXiv:1803.05957, 2018.
［21］	SHAHCHERAGHI Z, SEE J. On the effects of pre- and post-processing in video cartoonization with bilateral filters［C］// Proceedings of the 2013 IEEE International Conference on Signal and Image Processing Applications. 2013:37-42．
［22］	HUANG X, BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. 2017:1510-1519.
［23］	KINGMA D P, BA J L. Adam: A method for stochastic optimization［J］. arXiv preprint arXiv:1412.6980, 2014.
［24］	KARRAS T, AILA T, LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation［C］// Proceedings of the 2018 IEEE International Conference on Learning Representations. 2018:324-330.
［25］	WANG Z, BOVIK A C. A universal image quality index［J］. IEEE Signal Processing Letters, 2002,9(3):81-88.
［26］	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity［J］. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2004,13(4):600-612.
［27］	HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium［C］// Proceedings of the 31th International Conference on Neural Information Processing Systems. 2017:6629-6640.

[1]	付鸿林, 张太红, 杨雅婷, 艾孜麦提·艾瓦尼尔, 马博. 基于生成对抗网络的维语场景文字修改网络[J]. 计算机与现代化, 2024, 0(01): 41-46.
[2]	胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.
[3]	林威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6.
[4]	梁天恺, 黄康华, 刘凯航, 兰岚, 曾碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40.
[5]	邱凯星, 冯广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47.
[6]	张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75.
[7]	马泽宇, 叶宁, 徐康, 王甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107.
[8]	李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64.
[9]	黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76.
[10]	王鑫, 肖韬睿. 基于生成对抗网络的人脸识别对抗攻击[J]. 计算机与现代化, 2023, 0(10): 115-120.
[11]	沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9.
[12]	刘禅奕, 黄丹, 薛林雁, 王涛, 朱桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99.
[13]	马国祥, 杨凌菲, 严传波, 张志豪, 孙彬, 王晓荣. 基于深度DenseNet网络的肝包虫病超声影像诊断方法[J]. 计算机与现代化, 2023, 0(09): 100-104.
[14]	农皓程, 任德均, 任秋霖, 刘澎笠, 黄德成. 基于改进ConvNeXt的软塑包装表面异常检测算法[J]. 计算机与现代化, 2023, 0(08): 12-17.
[15]	欧阳飞, 吴旭, 向东升. 基于改进YOLOX的垃圾分类检测方法[J]. 计算机与现代化, 2023, 0(08): 68-73.