Multi-stage Rain Removal Algorithm Based on Multi-scale Frequency Attention

doi:10.3969/j.issn.1006-2475.2024.02.008

Abstract

Abstract: Abstract：Rain streaks interfere with the photos the outside vision system takes in rainy weather， which caused lowering the image quality and affecting the subsequent vision tasks. Therefore， for the following computer vision tasks， it is very crucial to remove the rain streaks from the photos and get high-quality images. The goal of the multi-stage rain removal method we present in this research is to recover a high quality image by removing rain streaks from a single rain image using multi-scale frequency attention. Firstly， a multi-stage rain removal model is designed by integrating the variety of rain streaks， decomposing the rain streaks removal process into multiple sub-processes， and eliminating rain streaks step by step. Second， a long and short-term memory recurrent network is improved to achieve multi-stage rain streaks removal， in which the frequency attention mechanism is introduced to strengthen the attention to rain streaks and a multi-scale feature extraction method is designed to characterize the global information. This addresses the issue of oversmoothing in the current rain streaks removal algorithms. The detail restoration module’s final purpose is to fortify background elements. Experiment results show that the proposed algorithm can effectively remove rain streaks on both the synthetic data set and the real dataset while preserving complete background information， and has a good rain removal effect.

Key words: Key words：single image rain removal, frequency attention, convolutional networks, recurrent neural network, discrete cosine transform

CLC Number:

TP391

WU Tian-tian, LI Yan-kai, LIU Yang. Multi-stage Rain Removal Algorithm Based on Multi-scale Frequency Attention[J]. Computer and Modernization, 2024, 0(02): 50-55.

References

［1］郭继昌，郭昊，郭春乐. 多尺度卷积神经网络的单幅图像去雨方法［J］. 哈尔滨工业大学学报， 2018，50（3）：185-191.
［2］陈舒曼，陈玮，尹钟. 单幅图像去雨算法研究现状及展望［J］. 计算机应用研究， 2022，39（1）：9-17.
［3］庞彦伟，周俊，邓君坪，等. 基于图像分解与字典分类的单幅图像去雨算法［J］. 天津大学学报（自然科学与工程技术版）， 2017，50（4）：391-398.
［4］胡明娣，吴怡，宋尧，等. 单幅图像去雨数据集和深度学习算法的联合评估与展望［J］. 中国图象图形学报， 2022，27（5）：1359-1391.
［5］ FU X Y， HUANG J B， DING X H， et al. Clearing the skies： A deep network architecture for single-image rain removal［J］. IEEE Transactions on Image Processing， 2017，26（6）：2944-2956.
［6］ HE K M， SUN J， TANG X O. Guided image filtering［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013，35（6）：1397-1409.
［7］ HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2016：770-778.
［8］ FU X Y， HUANG J B， ZENG D L， et al. Removing rain from single images via a deep detail network［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：1715-1723.
［9］ LI X， WU J L， LIN Z C， et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining［C］// Computer Vision – ECCV 2018. 2018：262-277.
［10］ HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018：7132-7141.
［11］ BALLAKUR A A， ARYA A. Empirical evaluation of gated recurrent neural network architectures in aviation delay prediction［C］// 2020 5th International Conference on Computing， Communication and Security （ICCCS）. 2020. DOI: 10.1109/ICCCS49678.2020.9276855.
［12］ FU X Y， LIANG B R， HUANG Y， et al. Lightweight pyramid networks for image deraining［J］. IEEE Transactions on Neural Networks and Learning Systems， 2020，31（6）：1794-1807.
［13］常敏，陈果，韩帅. 基于拉普拉斯金字塔的图像压缩与重构研究［J］. 包装工程， 2020，41（15）：239-244.
［14］彭延军，王瑾瑾，王元红. 基于拉普拉斯金字塔改进的图像融合方法［J］. 软件导刊， 2016，15（1）：167-170.
［15］ WEI Y Y， ZHANG Z， WANG Y， et al. DerainCycleGAN： Rain attentive CycleGAN for single image deraining and rainmaking［J］. IEEE Transactions on Image Processing， 2021，30：4788-4801.
［16］ ZHU J Y， PARK T， ISOLA P， et al. Unpaired image-to-image translation using cycle-consistent adversarial networks［C］// 2017 IEEE International Conference on Computer Vision （ICCV）. 2017：2242-2251.
［17］ WANG H， XIE Q， ZHAO Q， et al. RCDNet： An interpretable rain convolutional dictionary network for single image deraining［J］. IEEE Transactions on Neural Networks and Learning Systems， 2023. DOI: 10.1109/TNNLS.2022.3231453.
［18］王道累，张天宇. 图像去雾算法的综述及分析［J］. 图学学报， 2020，41（6）：861-870.
［19］ WANG R S， LI Z， CAO J， et al. Convolutional recurrent neural networks for text classification［C］// 2019 International Joint Conference on Neural Networks （IJCNN）. 2019. DOI: 10.1109/IJCNN.2019.8852406.
［20］ YANG Y， GUAN J W， HUANG S Y， et al. End-to-End rain removal network based on progressive residual detail supplement［J］. IEEE Transactions on Multimedia， 2022，24：1622-1636.
［21］ REN D W， ZUO W M， HU Q H， et al. Progressive image deraining networks： A better and simpler baseline［C］// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：3932-3941.
［22］ YANG W H， TAN R T， FENG J S， et al. Deep joint rain detection and removal from a single image［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：1685-1694.
［23］张爱苓，李少荣，张瑞. 结合暗通道与图像分层的图像去雨雾方法［J］. 工业控制计算机， 2022，35（12）：67-69.
［24］ QIN Z Q， ZHANG P Y， WU F， et al. FcaNet： Frequency channel attention networks［C］// 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021：763-772.
［25］汪帆，魏宪，郭杰龙，等. 基于多通道分离整合的多尺度单幅图像去雨算法［J］. 计算机与现代化， 2021（12）：72-78.
［26］程安琪. 基于多尺度特征的单幅图像去雨算法研究［D］. 上海：上海师范大学， 2023.
［27］ DU Y， DENG J J， ZHENG Y L， et al. DSDNet： Toward single image deraining with self-paced curricular dual stimulations［J］. Computer Vision and Imzage Understanding. 2023，230. DOI: 10.1016/j.cviu.2023.103657.
［28］刘彦清. 基于YOLO系列的目标检测改进算法［D］. 长春：吉林大学， 2021.

[1]	SUN Zi-yu, REN Ran, WEI Xi-zhe. Research on Stock Classification and Forecast Based on DTW-TCN [J]. Computer and Modernization, 2023, 0(08): 31-37.
[2]	YIN Chun-jie, XIAO Fa-da, LI Peng-fei, ZHAO Qin. Short-term Load Forecasting of Regional Microgrid Based on LSTM Neural Network [J]. Computer and Modernization, 2022, 0(04): 7-11.
[3]	WANG Yu-ying, WANG Yong. Recommendation Algorithm Based on Knowledge Graph and Bi-LSTM [J]. Computer and Modernization, 2021, 0(09): 90-98.
[4]	GAO Yi-fan, WANG Yong. An Image Description Algorithm Based on Object Detection and Part of Speech Analysis [J]. Computer and Modernization, 2021, 0(03): 108-114.
[5]	QI Yu-dong1, DING Hai-qiang1, ZHAO Jin-chao2, SUN Ming-wei1. biRNN-based Method for Processing Unbalanced Text Data Sets of Naval Ordnance [J]. Computer and Modernization, 2019, 0(12): 21-.
[6]	Gengzang-Cuomao1,2, HUANG He-ming1,2. Application of Bidirectional Recurrent Neural Network in Speech Recognition [J]. Computer and Modernization, 2019, 0(10): 1-.
[7]	DAI Hai-neng, MAO Yao-bin. An Improved Face Detection Algothrim Based on R-FCN [J]. Computer and Modernization, 2018, 0(08): 12-.
[8]	YE Xiao-zhou, TAO Fei-fei, QI Rong-zhi, ZHANG Yun-fei, ZHOU Si-qi, LIU Xuan. Improvement on Activation Functions of Recurrent Neural Network Architectures [J]. Computer and Modernization, 2016, 0(12): 29-33.
[9]	XIN Xin1,3, CHEN Shudong2, TONG Minglei4, HU Wenhao1,3, LIU Chenwei1,3, GE Haodong3. An Improved Sound Signal Compression Algorithm Based on Discrete Cosine Transform [J]. Computer and Modernization, 2016, 0(10): 1-5.
[10]	DOU Yong-mei. An Embedded Algorithm for Gray Watermarking Image Based on DWT and DCT [J]. Computer and Modernization, 2014, 0(2): 197-200.
[11]	ZHENG Bo-chuan. A Lotka-Volterra Recurrent Neural Network Model for Solving TSP [J]. Computer and Modernization, 2013, 1(8): 204-208.
[12]	LAI Li;DING Jin-song. A Digital Watermarking Algorithm Based on Discrete Cosine Transform [J]. Computer and Modernization, 2012, 1(6): 172-175.
[13]	WANG Zongli. Realization and Comparison of Two Kinds of Transformations Territories Digital Watermark Algorithm  [J]. Computer and Modernization, 2009, 169(9): 87-91.