基于全变分正则和L2,1范数的视频去雨张量模型

doi:10.3969/j.issn.0255-8297.2022.02.006

摘要/Abstract

摘要： 提出了一种基于全变分正则与$L_{2，1}$范数的视频去雨张量模型用于解决雨线遮挡问题。首先，对雨线成分与视频背景先验信息进行预处理，获取相应正则化条件的构建依据以增强各部分稀疏性，便于促进雨线分离。其次，考虑到视频图像存在不规则动态对象，引入全变分正则项来抑制背景强度变化，缓解雨线的误判现象。采用交替方向乘子法（alternating direction method of multipliers，ADMM）可以有效地对所提出的张量模型进行求解，并在合成数据与真实数据集上开展大量实验。结果表明，所提方法在动态背景情况下有效去除视频图像雨线的同时，保留了更多背景细节信息。与相关先进方法相比，所提方法在峰值信噪比、结构相似性和残差三种综合性能量化指标上均具有较大的优势。

关键词: 视频去雨, 张量模型, 范数, 全变分, 交替方向乘子法

Abstract: This paper proposes a tensor model for video rain streaks removal based on total variational regularization and L_2,1 norm to solve rain streaks shielding. Firstly, the prior information of rain streaks component and the video background are preprocessed to obtain the corresponding regularization condition, so as to enhance the sparsity of each part and facilitate the separation of rain streaks. Then, considering the existence of irregular dynamic objects in the video, a total variational regularization term is introduced to suppress the variation of background intensity and alleviate the misjudgment of rain streaks. The alternating direction method of multipliers (ADMM) can be used to effectively solve the proposed tensor model, and carried out a large number of experiments on the synthetic and real datasets. Experimental results show that the proposed method under dynamic background can effectively remove the video rain streaks and retain more background details simultaneously. Compared with other relevant methods, the proposed method has great advantages in three comprehensive quantitative performance measures of peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and residual error (RES).

Key words: rain removal, tensor modeling, norm, total variation regularization, alternating direction method of multipliers(ADMM)

中图分类号:

卢星含, 郑钰辉, 张建伟. 基于全变分正则和L_2,1范数的视频去雨张量模型[J]. 应用科学学报, 2022, 40(2): 233-245.

LU Xinghan, ZHENG Yuhui, ZHANG Jianwei. Tensor Model Based on Total Variation Regularized and L_2,1 Norm for Video Rain Streaks Removal[J]. Journal of Applied Sciences, 2022, 40(2): 233-245.

参考文献

[1] Zhu Q S, Shao L, Heng P A, et al. A novel rain detection and removal approach using guided filtering and formation modeling[C]//2013 IEEE International Conference on Robotics and Biomimetics, 2013:563-567.
[2] Wang C, Shen M M, Yao C. Rain streak removal by multi-frame-based anisotropic filtering[J]. Multimedia Tools and Applications, 2017, 76(2):2019-2038.
[3] Barnum P, Kanade T, Narasimhan S G. Spatio-temporal frequency analysis for removing rain and snow from videos[C]//International Workshop on Photometric Analysis For Computer Vision, 2007:1-8.
[4] Santhaseelan V, Asari V K. Utilizing local phase information to remove rain from video[J]. International Journal of Computer Vision, 2015, 112(1):71-89.
[5] Chen Z, Shen J H. A new algorithm of rain (snow) removal in video[J]. Journal of Multimedia, 2013, 8(2):168-174.
[6] Fadili M J, Starck J L, Bobin J, et al. Image decomposition and separation using sparse representations:an overview[J]. Proceedings of the IEEE, 2010, 98(6):983-994.
[7] Kang L W, Lin C W, Lin C T, et al. Self-learning-based rain streak removal for image/video[C]//2012 IEEE International Symposium on Circuits and Systems, 2012:1871-1874.
[8] Sun S H, Fan S P, Wang Y C F. Exploiting image structural similarity for single image rain removal[C]//2014 IEEE International Conference on Image Processing, 2014:4482-4486.
[9] Ramya C, Subha Rani S. Rain removal in image sequence using sparse coding[M]//Communications in Computer and Information Science. Berlin, Heidelberg:Springer Berlin Heidelberg, 2012:361-370.
[10] Zheng X H, Liao Y H, Guo W, et al. Single-image-based rain and snow removal using multiguided filter[M]//Neural Information Processing. Berlin, Heidelberg:Springer Berlin Heidelberg, 2013:258-265.
[11] Wang Y L, Chen C, Zhu S Y, et al. A framework of single-image deraining method based on analysis of rain characteristics[C]//2016 IEEE International Conference on Image Processing, 2016:4087-4091.
[12] Chen D Y, Chen C C, Kang L W. Visual depth guided color image rain streaks removal using sparse coding[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(8):1430-1455.
[13] Zhou Y, Han Y S, Zhou P C. A method for removing raindrops from a single image[J]. Journal of Graphics, 2015, 36(3):438-443.
[14] Li Y, Tan R T, Guo X J, et al. Rain streak removal using layer priors[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016:2736-2744.
[15] Chen Y L, Hsu C T. A generalized low-rank appearance model for spatio-temporally correlated rain streaks[C]//2013 IEEE International Conference on Computer Vision, 2013:1968-1975.
[16] Kim J H, Sim J Y, Kim C S. Video deraining and desnowing using temporal correlation and low-rank matrix completion[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2015, 24(9):2658-2670.
[17] Okatani T, Yoshida T, Deguchi K. Efficient algorithm for low-rank matrix factorization with missing components and performance comparison of latest algorithms[C]//2011 International Conference on Computer Vision, 2011:842-849.
[18] Jiang T X, Huang T Z, Zhao X L, et al. A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017:2818-2827.
[19] Li M H, Xie Q, Zhao Q, et al. Video rain streak removal by multiscale convolutional sparse coding[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018:6644-6653.
[20] Sun Z Y, Xiong S W, Liu R W. Directional regularized tensor modeling for video rain streaks removal[DB/OL]. 2019[2020-10-14]. https://arxiv.org/abs/1902.07090.
[21] Wang Y T, Zhao X L, Jiang T X, et al. A total variation and group sparsity based tensor optimization model for video rain streak removal[J]. Signal Processing:Image Communication, 2019, 73:96-108.
[22] Yang W H, Tan R T, Feng J S, et al. Deep joint rain detection and removal from a single image[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017:1685-1694.
[23] Fu X Y, Huang J B, Ding X H, et al. Clearing the skies:a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2017, 26(6):2944-2956.
[24] Liu J Y, Yang W H, Yang S, et al. D3R-net:dynamic routing residue recurrent network for video rain removal[J]. IEEE Transactions on Image Processing, 2019, 28(2):699-712.
[25] Liu R S, Jiang Z Y, Ma L, et al. Deep layer prior optimization for single image rain streaks removal[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing, 2018:1408-1412.
[26] Guo X J, Ma Y. Generalized Tensor Total Variation minimization for visual data recovery?[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015:3603-3611.
[27] Jiang Y Q, Jin X, Wu Z Y. Video inpainting based on joint gradient and noise minimization[C]//Advances in Multimedia Information Processing-PCM 2016, 2016:407-417.
[28] Liu R W, Shi L, Huang W H, et al. Generalized total variation-based MRI Rician denoising model with spatially adaptive regularization parameters[J]. Magnetic Resonance Imaging, 2014, 32(6):702-720.
[29] Rudin L I, Osher S, Fatemi E. Nonlinear total variation based noise removal algorithms[J]. Physica D:Nonlinear Phenomena, 1992, 60(1/2/3/4):259-268.
[30] Chan S H, Khoshabeh R, Gibson K B, et al. An augmented Lagrangian method for total variation video restoration[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2011, 20(11):3097-3111.
[31] Nikolova M. An algorithm for total variation minimization and applications[J]. Journal of Mathematical Imaging and Vision, 2004, 20(1/2):89-97.
[32] Cao X C, Yang L, Guo X J. Total variation regularized RPCA for irregularly moving object detection under dynamic background[J]. IEEE Transactions on Cybernetics, 2016, 46(4):1014-1027.
[33] Boyd S, Parikh N, Chu E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends in Machine Learning, 2010, 3(1):1-122.
[34] Deng W, Yin W T, Zhang Y. Group sparse optimization by alternating direction method[C]//SPIE Optical Engineering Applications. San Diego, California, 2013, 8858:242-256.