An Image Style Conversion Technology Based on EBGAN

doi:10.3969/j.issn.1006-2475.2020.04.005

Abstract

Abstract: In order to solve the problem of poor diversity of the generated images in the traditional image style conversion algorithm, this paper proposes a network model based on EBGAN (Energy-Based Generative Adversarial Net). The idea of energy function is introduced into the discriminator, and the Autoencoder is designed to generate different reconstruction results for the true and false input respectively. The error value before and after the reconstruction of the input image is calculated, which is used as the energy concept to identify the input image. In the coding stage of Autoencoder, the orthogonal control is introduced in to the encoded vectors to control the maximum orthogonalization of two vectors in the same batch, so as to promote the generator net to generate images in different directions. Experiments on Facades and Cityscapes datasets show that the proposed network model can effectively achieve process of image stylization and generate more diversified images than the traditional network model.

Key words: GAN, energy function, image style conversion

CLC Number:

TP183

TAO Ying, LIU Hui-yi. An Image Style Conversion Technology Based on EBGAN[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2020.04.005.

References

［1］ GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style［J］. Computer Vision and Pattern Recognition, 2015:arXiv:1508.06576.
［2］陈淑環,韦玉科,徐乐,等. 基于深度学习的图像风格迁移研究综述［J］. 计算机应用研究, 2019,36(8):2250-2255.
［3］ GATYS L, ECKER A S, BETHGE M. Texture synthesis using convolutional neural networks［C］// Advances in Neural Information Processing Systems. 2015:262-270.
［4］ LI Y H, WANG N Y, LIU J Y, et al. Demystifying neural style transfer［J］. Computer Vision and Pattern Recognition, 2017:arXiv:1701.01036.

［5］ GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014:2672-2680.
［6］ ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks［C］// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:1125-1134.
［7］ CHEN Q, KOLTUN V. Photographic image synthesis with cascaded refinement networks［C］// Proceedings of 2017 IEEE International Conference on Computer Vision. 2017:1511-1520.
［8］ ZHU J Y, ZHANG R, PATHAK D, et al. Toward multimodal image-to-image translation［C］// Advances in Neural Information Processing Systems. 2017:465-476.
［9］ ZHAO J, MATHIEU M, LECUN Y. Energy-based generative adversarial network［J］. Machine Learning, 2016:arXiv:1609.03126.
［10］RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// International Conference on Medical Image Computing and Computer-assisted Intervention. 2015:234-241.
［11］VINCENT P, LAROCHELLE H, LAJOIE I, et al. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion［J］. Journal of Machine Learning Research, 2010,11(12):3371-3408.
［12］YAMANAKA J, KUWASHIMA S, KURITA T. Fast and accurate image super resolution by deep CNN with skip connection and network in network［C］// International Conference on Neural Information Processing. 2017:217-225.
［13］KAVUKCUOGLU K, SERMANET P, BOUREAU Y L, et al. Learning convolutional feature hierarchies for visual recognition［C］// Advances in Neural Information Processing Systems. 2010:1090-1098.
［14］IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing Internal covariate shift［C］// Proceedings of the 32nd International Conference on Machine Learning. 2015:448-456.
［15］KIM T, BENGIO Y. Deep directed generative models with energy-based probability estimation［J］. Machine Learning, 2016:arXiv:1606.03439.
［16］WANG R, CULLY A, CHANG H J, et al. Magan: Margin adaptation for generative adversarial networks［J］. Machine Learning, 2017:arXiv:1704.03817.
［17］RATLIFF L J, BURDEN S A, SASTRY S S. Characterization and computation of local Nash equilibria in continuous games［C］// Proceedings of 2013 51st Annual Allerton Conference on Communication, Control, and Computing. 2013:917-924.
［18］LIU J W, CHI G H, LUO X L. Contrastive divergence learning for the restricted Boltzmann machine［C］// 2013 9th International Conference on Natural Computation. 2013:18-22.
［19］HYNES M B, DE STERCK H. A polynomial expansion line search for large-scale unconstrained minimization of smooth L 2-regularized loss functions, with implementation in Apache Spark［C］// Proceedings of 2016 SIAM International Conference on Data Mining. 2016:603-611.
［20］ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein gan［J］. Machine Learning, 2017:arXiv:1701.07875.
［21］GOLDSTEIN T, OSHER S. The split Bregman method for L1-regularized problems［J］. SIAM Journal on Imaging Sciences, 2009,2(2):323-343.
［22］KINGMA D P, BA J. Adam: A method for stochastic optimization［J］. Machine Learning, 2014:arXiv:1412.6980.
［23］HUYNH-THU Q, GHANBARI M. Scope of validity of PSNR in image/video quality assessment［J］. Electronics Letters, 2008,44(13):800-801.

[1]	FU Hong-lin, ZHANG Tai-hong, YANG Ya-ting, Aizimaiti Aiwanier, MA Bo. Scenes Text Modification Network for Uyghur Based on Generative Adversarial Network [J]. Computer and Modernization, 2024, 0(01): 41-46.
[2]	WANG Jia-chen, ZHANG Hong-xin, LIU Qing-hua, . CT Image Generation of Pneumonia Based on Generative Adversarial Network [J]. Computer and Modernization, 2023, 0(07): 20-24.
[3]	YIN Xiang-chen, CHEN Si-long, LI Zhen-kai, ZHANG Wen-jin, LI Gui-qing. Algorithm of Multi-scale Dense Receptive Domain GAN lmage Dehazing [J]. Computer and Modernization, 2023, 0(03): 11-15.
[4]	LI Hai-tao, HU Ze-tao, ZHANG Jun-hu. Method of Fish Image Expansion Based on NS-StyleGAN2 Network [J]. Computer and Modernization, 2023, 0(01): 13-17.
[5]	PENG Peng-fei, ZHOU Lin-ru. GRU Adversarial Network Text Generation Model with Reward [J]. Computer and Modernization, 2022, 0(07): 121-126.
[6]	JIN Long, WU You, ZHANG Yong-xiang. OFDM Channel Estimation Based on Improved SRGAN [J]. Computer and Modernization, 2021, 0(10): 112-118.
[7]	JIANG Jing-jie, XU Luo, LI Ning. Method of Metamorphic Testing for Image Recognition System Based on GAN [J]. Computer and Modernization, 2021, 0(02): 24-29.
[8]	HAN Can-can, LI Zhi-hua, XU Rui. Method of Nonparallel Speech Denoising Based on CycleGAN [J]. Computer and Modernization, 2021, 0(02): 73-77.
[9]	SUN Zhi-bo, XU Xiang-hui, . Simulation Method of Target SAR Image Based on Spectral Normalization Generative Adversarial Network [J]. Computer and Modernization, 2020, 0(08): 14-20.
[10]	JI Yan-ling1, LIN Zhi-xian1, TANG Qian1, GUO Tai-liang1, TANG Biao2. OLED Defective Pixel Recognition Based on K-means Clustering and LVQ Neural Network [J]. Computer and Modernization, 2019, 0(07): 37-.
[11]	XIAO Wen-guo1, GE Hua-yong1,2, PENG Yang-yang1. A Color Image Steganography Algorithm Based on Modulus Computation [J]. Computer and Modernization, 2018, 0(10): 48-.
[12]	XU Cong, HUANG Wen-zhun, HUANG Shi-qi. Hyper-heuristic Genetic Cluster Algorithm Based on Self-organizing Map [J]. Computer and Modernization, 2017, 0(4): 38-43.
[13]	SHEN Xiajiong1， CHEN Wei1,2， HUANG Xiangzhi2， ZANG Wenqian2. Research on Organization Mode of Satellite Quality#br# Evaluation Module Based on Multidimensional Data Model [J]. Computer and Modernization, 2017, 0(3): 101-.
[14]	FAN Yi-na, LANG Bo, HUANG Jing. An Image Representation Computation Model Based on Dynamic Adjustment Mechanism of Non-classical Receptive Fields [J]. Computer and Modernization, 2015, 0(8): 13-18.
[15]	JIANG Kai, YUAN Jin-hai. A Novel Assembled Text Clustering Algorithm Using Differential Evolution and SOM [J]. Computer and Modernization, 2015, 0(5): 13-16+20.