Arbitrary Style Transfer Method with Multi-scale Semantic Adaptation

doi:10.3969/j.issn.1006-2475.2025.09.011

Abstract

Abstract:
Abstract: Addressing the challenge of balancing style and content information in existing arbitrary style transfer models， this paper introduces an improved model for arbitrary style transfer. The model incorporates a multi-scale semantic adjustment module before the style transformation process. This module deeply adjusts style and content feature mappings to enhance key feature expressions， improving the coherence of image content structure and style features after style transfer. Additionally， a semantic adjustment loss function is proposed to precisely preserve the original image’s content structure and delicately transfer the target style’s features. The experimental results show that this method not only maintains the content information but also enhances the style transfer effects.

Key words: Key words: arbitrary style transfer, multi-scale semantic adjustment, enhance key feature expressions, semantic adjustment loss

CLC Number:

中图分类号：TP391

ZHU Lulu, GU Lin. Arbitrary Style Transfer Method with Multi-scale Semantic Adaptation[J]. Computer and Modernization, 2025, 0(09): 73-78.

References

［1］刘冰. 基于卷积神经网络的图像风格迁移算法综述［J］. 新疆师范大学学报（自然科学版）， 2022，41（4）:1-7.
［2］廉露，田启川，谭润，等. 基于神经网络的图像风格迁移研究进展［J］. 计算机工程与应用， 2024，60（9）:30-47.
［3］ GATYS L A， ECKER A S， BETHGE M. A neural algorithm of artistic style［J］. arXiv preprint arXiv:1508.06576，
2015.
［4］ JOHNSON J， ALAHI A， FEI-FEI L. Perceptual losses for real-time style transfer and super-resolution［C］// Proceedings of 2016 14th European Conference on Computer Vision. Springer， 2016:694-711.
［5］ HUANG X， BELONGIE S. Arbitrary style transfer in real-time with adaptive instance normalization［C］// Proceedings of 2017 IEEE International Conference on Computer Vision. IEEE， 2017:1501-1510.
［6］ PARK D Y， LEE K H. Arbitrary style transfer with style-attentional networks［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2019:5880-5888.
［7］ ZHANG Y B， LI M H， LI R H， et al. Exact feature distribution matching for arbitrary style transfer and domain generalization［C］// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2022:8035-8045.
［8］ DENG Y Y， TANG F， DONG W M， et al. Arbitrary style transfer via multi-adaptation network［C］// Proceedings of the 28th ACM International Conference on Multimedia. ACM， 2020:2719-2727.
［9］ GOODFELLOW I， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial networks［J］. Communications of the ACM， 2020，63（11）:139-144.
［10］ ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［C］// Proceedings of 2017 IEEE International Conference on Computer Vision. IEEE， 2017:2223-2232.
［11］ CHEN Y， LAI Y K， LIU Y J. CartoonGAN: Generative adversarial networks for photo cartoonization［C］// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2018:9465-9474.
［12］ KARRAS T， LAINE S， AILA T. A style-based generator architecture for generative adversarial networks［C］// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2019:4401-4410.
［13］ YANG S， JIANG L M， LIU Z W， et al. Pastiche master: Exemplar-based high-resolution portrait style transfer［C］// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2022:7693-7702.
［14］ LI Y J， FANG C， YANG J M， et al. Universal style transfer via feature transforms［J］. arXiv preprint arXiv:1705.08086，
2017.
［15］ SHENG L， LIN Z Y， SHAO J， et al. Avatar-Net: Multi-scale zero-shot style transfer by feature decoration［C］// Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2018:8242-8250.
［16］ JING Y C， LIU X， DING Y K， et al. Dynamic instance normalization for arbitrary style transfer［J］. AAAI Conference on Artificial Intelligence， 2020，34（4）:4369-4376.
［17］ CHANDRAN P， ZOSS G， GOTARDO P， et al. Adaptive convolutions for structure-aware style transfer［C］// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2021:7972-7981.
［18］ LI W T， RAHUL V， RAMYA S P. Artistic arbitrary style transfer［J］. arXiv preprint arXiv: 2212.11376， 2022.
［19］ DUMOULIN V， SHLENS J， KUDLUR M. A learned representation for artistic style［J］. arXiv preprint arXiv:1610.
07629， 2016.
［20］祁宣豪，智敏. 图像处理中注意力机制综述［J］. 计算机科学与探索， 2024，18（2）:345-362.
［21］米志鹏. 基于注意力和特征分布匹配的图像风格迁移方法研究［D］. 马鞍山:安徽工业大学， 2022.
［22］杨玥，冯涛，梁虹，等. 融合交叉注意力机制的图像任意风格迁移［J］. 计算机科学， 2022，49（增1）:345-352+396.
［23］谭润，田启川，廉露，等. 融合超分辨率重构的图像任意风格迁移［J］. 计算机工程与应用， 2024，60（15）:170-179.
［24］李鑫，普园媛，赵征鹏，等. 艺术美感增强的图像任意风格迁移［J］. 计算机科学， 2024，51（9）:129-139.
［25］张彩灯，徐杨，莫寒，等. 融合多注意力机制的语义调整风格迁移网络［J］. 计算机工程与应用， 2025，61（8）:204-214.
［26］ SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556， 2014.
［27］ OUYANG D L， HE S， ZHANG G Z， et al. Efficient multi-scale attention module with cross-spatial learning［C］// ICASSP 2023-2023 IEEE International Conference on Acoustics， Speech and Signal Processing （ICASSP）. IEEE，
2023:1-5.
［28］ YEO C， TAN H L， TAN Y H. On rate distortion optimization using SSIM［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2013，23（7）:1170-1181.
［29］ IOANNOU E， STEVE M. Valuation in neural style transfer: A review［J］. arXiv preprint arXiv:2401.17109， 2024.

[1]	LUO Deyan, XU Yang, ZUO Fengyun, ZHANG Yongdan. Lightweight Garbage Classification and Detection of Improved YOLOv8 [J]. Computer and Modernization, 2025, 0(09): 35-42.
[2]	LIN Ruizi, YAO Da, DAI Xin, SHEN Guoyu, WANG Jiahui, WAN Weiguo. Facial Sketch Image Conversion Based on CycleGAN and Attention Mechanism [J]. Computer and Modernization, 2025, 0(09): 61-66.
[3]	YANG Tianshun1, SONG Huansheng1, LIANG Haoxiang2, LIU Haonan1, MA Xinzhou1, SUN Shijie1, ZHANG Shaoyang1. Multimodal Tunnel Fire Detection Based on Temporal Features of Video [J]. Computer and Modernization, 2025, 0(09): 79-89.
[4]	LI Su, LI Dequan, QING Yu. Single View 3D Hair Modeling Based on Mask Constraints [J]. Computer and Modernization, 2025, 0(09): 90-96.
[5]	LIU Cheng, FENG Guang. CNN-BiLSTM and LightGBM Stock Prediction Based on Dual Attention Mechanism [J]. Computer and Modernization, 2025, 0(09): 97-103.
[6]	SHANG Boxiang1, GUO Xiaoyan2, ZHENG Jian1, SUN Xianfan2. Joint Knowledge Extraction for Cloud-edge Collaborative Multi-source Transmission Data [J]. Computer and Modernization, 2025, 0(09): 104-108.
[7]	CHENG Mengyuan, LI Yanhui, LYU Tianci, ZHAO Yuxin, HUANG Chen. FedLDP： Efficient Federated Learning with Localized Differential Privacy [J]. Computer and Modernization, 2025, 0(09): 109-118.
[8]	SHI Fengyuan1, 2, MAO Yi3, JIAO Lei4. Cross-domain Book Recommendation Method Based on Heterogeneous Information Network [J]. Computer and Modernization, 2025, 0(08): 24-30.
[9]	LI Yan, MAO Jiaming, WANG Ziying, GU Zhimin, JIANG Haitao. Anomalous Event Prediction Approach Using Graph Neural Network [J]. Computer and Modernization, 2025, 0(08): 39-47.
[10]	CHENG Yuwen, JING Yijun, SHI Zicheng, JING Changqiang, GUO Feng, WU Chuankun. A Data Privacy Protection Scheme Based on Federated Learning [J]. Computer and Modernization, 2025, 0(08): 63-69.
[11]	GAO Lisha1, PENG Guozheng2, WANG Chong1, HAN Shuo1, XIANG Nan1, ZHANG Qizhe2, ZHU Yuefei3, CHENG Xu3. Dual-backbone Network Insulator Defect Detection Algorithm Based on YOLOv5 [J]. Computer and Modernization, 2025, 0(08): 82-88.
[12]	HU Shaowen, HUANG Langxin, YU Lihui, SHI Weili, JIANG Nan, SUN Wu, LUO Shuhuan. Optimizing Wavelet Neural Network for Pure Milk Recognition using Improved Genetic Algorithm [J]. Computer and Modernization, 2025, 0(08): 97-103.
[13]	LI Gang, SUN Shaobin, KONG Wei, ZHAO Lei. Research on Multi-objective Coordinated Control Method of User-side Energy Storage Based on Harmony System [J]. Computer and Modernization, 2025, 0(08): 119-126.
[14]	SUN Kaijie, HU Jili. Application of Multimodal Large Language Models in Diagnosis of Pigmented Skin Lesions [J]. Computer and Modernization, 2025, 0(07): 21-27.
[15]	HE Lili1, 2, ZHANG Chenglin1, 2, ZHANG Lei1, 2 , CAO Mingzeng1, 2. Double Privacy Protection Algorithm for User Location Based on Multi-anonymizer Architecture [J]. Computer and Modernization, 2025, 0(07): 33-42.