基于NSCT与压缩感知的红外影像融合

doi:10.3969/j.issn.0255-8297.2022.01.008

摘要/Abstract

摘要： 针对红外和可见光图像在融合过程中存在质量低下、信息缺失、边缘细节不突出等问题，提出一种基于非下采样轮廓波变换（non-subsampled contourlet transform，NSCT）与稀疏表示的压缩感知图像融合重构算法。首先利用NSCT进行源图像分解，得到相应的高频子带和低频子带图像；然后针对高频子带部分，利用基于压缩感知的高频融合规则进行融合，得到高频融合系数；针对低频子带部分，按照基于字典学习的低频融合规则进行融合，得到低频融合系数。最后进行NSCT逆变换得到融合影像，实现红外和可见光图像的超分辨率恢复。实验结果表明：采用该算法融合后的图像在平均梯度、边缘强度、信息熵、边缘信息保留度、空间频率等指标上均有良好的表现，体现出该融合算法在图像融合质量的提升方面颇具优势。

关键词: 图像融合, 非下采样轮廓波变换, 稀疏表示, 压缩感知, 红外影像

Abstract: Aiming at the problems of low quality, lack of information and non-prominent edge details in the fusion process of infrared and visible images, this paper proposes a compressed sensing image fusion and reconstruction algorithm based on non-subsampled contourlet transform (NSCT) and sparse representation. Firstly, a source image is decomposed by using NSCT to obtain corresponding high-frequency sub-band and low-frequency sub-band images. Then, the high-frequency sub-band images are fused by using the highfrequency fusion rules based on compressed sensing to obtain high-frequency fusion coefficients. For the low-frequency sub-band images, low-frequency fusion coefficients are obtained by using the low-frequency fusion rules based on dictionary learning. Finally, a fusion image is obtained by using the inverse NSCT transformation to achieve superresolution recovery of infrared and visible images. Experimental results show that the images fused by this algorithm have good performance in metrics, such as average gradient, edge intensity, information entropy, edge information retention and spatial frequency, and prove that this fusion algorithm has significant advantages in image fusion quality.

Key words: image fusion, non-subsampled contour transform (NSCT), sparse representation, compressed sensing (CS), infrared image

中图分类号:

TP751
TN219

金安安, 李祥, 张丽, 熊卿智. 基于NSCT与压缩感知的红外影像融合[J]. 应用科学学报, 2022, 40(1): 80-92.

JIN An'an, LI Xiang, ZHANG Li, XIONG Qingzhi. Infrared Image Fusion Based on NSCT and Compressed Sensing[J]. Journal of Applied Sciences, 2022, 40(1): 80-92.

参考文献

[1] Kaur H, Koundal D, Kadyan V. Image fusion techniques:a survey[J]. Archives of Computational Methods in Engineering, 2021:1-23.
[2] 陈清江, 李毅, 柴昱洲. 基于卷积神经网络的红外图像融合算法[J]. 激光与红外, 2019, 49(1):123-128. Chen Q J, Li Y, Chai Y Z. Infrared image fusion algorithm based on convolutional neural network[J]. Laser & Infrared, 2019, 49(1):123-128. (in Chinese)
[3] 甄媚. 可见光图像与红外图像融合算法研究[D]. 西安:西安科技大学, 2019.
[4] 冯颖, 贺兴时, 薛菁菁, 等. 基于NSCT的SAR与可见光图像融合算法[J]. 电光与控制, 2018, 25(3):23-27. Feng Y, He X S, Xue J J, et al. Fusion algorithm of SAR and visible image based on NSCT[J]. Electro-optical and Control, 2018, 25(3):23-27. (in Chinese)
[5] 罗高鹏. 基于多尺度变换的红外与可见光图像融合算法研究[D]. 合肥:合肥工业大学, 2017.
[6] 丁贵鹏, 陶钢, 李英超, 等. 基于非下采样轮廓波变换与引导滤波器的红外及可见光图像融合[J]. 兵工学报, 2021, 42(9):1911-1922. Ding G P, Tao G, Li Y C, et al. Infrared and visible light image fusion based on nonsubsampled contourlet transform and guided filter[J]. Acta Armamentarii, 2021, 42(9):1911-1922. (in Chinese)
[7] 杨孙运, 奚峥皓, 王汉东, 等. 基于NSCT和最小化-局部平均梯度的图像融合[J]. 红外技术, 2021, 43(1):13-20. Yang S Y, Xi Z H, Wang H D, et al. Image fusion based on NSCT and minimum-local mean gradient[J]. Infrared Technology, 2021, 43(1):13-20. (in Chinese)
[8] Yang Y, Ding M, Huang S, et al. Multi-focus image fusion via clustering PCA based joint dictionary learning[J]. IEEE Access, 2017, 5:16985-16997.
[9] Xia J, Chen Y, Chen A. Medical image fusion based on sparse representation and PCNN in NSCT domain[J]. Computational and Mathematical Methods in Medicine, 2018, 2018:1-12.
[10] Bhateja V, Patel H, Krishn A, et al. Multimodal medical image sensor fusion framework using cascade of wavelet and contourlet transform domains[J]. IEEE Sensors Journal, 2015, 15(12):6783-6790.
[11] Zhang Q, Guo B L. Research on image fusion based on the nonsubsampled contourlet transform[C]//2007 IEEE International Conference on Control and Automation, 2007:3239-3243.
[12] Da Cunha A L, Zhou J, Do M N. The nonsubsampled contourlet transform:theory, design, and applications[C]//IEEE Transactions on Image Processing, 2006, 15(10):3089-3101.
[13] Wang Z, Li X, Duan H, et al. Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform[J]. Expert Systems with Applications, 2021, 171:114574.
[14] Li X, Zhao J. A novel multi-modal medical image fusion algorithm[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(2):1995-2002.
[15] Olshausen B A, Field D J. Sparse coding with an overcomplete basis set:a strategy employed by V1?[J]. Vision Research, 1997, 37(23):3311-3325.
[16] Li Q, Wang W, Chen G, et al. Medical image fusion using segment graph filter and sparse representation[J]. Computers in Biology and Medicine, 2021, 131:104239.
[17] Maqsood S, Javed U. Multi-modal medical image fusion based on two-scale image decomposition and sparse representation[J]. Biomedical Signal Processing and Control, 2020, 57:101810.
[18] 余汪洋, 陈祥光, 董守龙, 等. 基于小波变换的图像融合算法研究[J]. 北京理工大学学报, 2014, 34(12):1262-1266. Yu W Y, Chen X G, Dong S L, et al. Research on image fusion algorithm based on wavelet transform[J]. Transactions of Beijing Institute of Technology, 2014, 34(12):1262-1266. (in Chinese)
[19] Jin X, Jiang Q, Yao S, et al. Infrared and visual image fusion method based on discrete cosine transform and local spatial frequency in discrete stationary wavelet transform domain[J]. Infrared Physics & Technology, 2018, 88:1-12.
[20] Liu Y, Chen X, Peng H, et al. Multi-focus image fusion with a deep convolutional neural network[J]. Information Fusion, 2017, 36:191-207.
[21] Liu Y, Chen X, Ward R K, et al. Medical image fusion via convolutional sparsity based morphological component analysis[J]. IEEE Signal Processing Letters, 2019, 26(3):485-489.
[22] Liu Y, Chen X, Ward R K, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12):1882-1886.
[23] Liu Y, Liu S, Wang Z. Multi-focus image fusion with dense SIFT[J]. Information Fusion, 2015, 23:139-155.
[24] Naji Ma, Aghagolzadeh A. Multi-focus image fusion in DCT domain based on correlation coefficient[C]//20152nd International Conference on Knowledge-Based Engineering and Innovation, 2015:632-639.
[25] Liang W, Long J, Li K C, et al. A fast defogging image recognition algorithm based on bilateral hybrid filtering[C]//ACM Transactions on Multimedia Computing, Communications, and Applications, 2021, 17(2):1-16.
[26] Tamilkodi R, Nesakumari G R. A novel framework for retrieval of image using weighted edge matching algorithm[J]. Multimedia Tools and Applications, 2021, 80(13):19625-19648.
[27] Jeon G. Information entropy algorithms for image, video, and signal processing[J]. Entropy, 2021, 23(8):926.
[28] Li X, Zhao J. A novel multi-modal medical image fusion algorithm[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(2):1995-2002.
[29] Liu C, Yang B, Li Y, et al. An information retention and feature transmission network for infrared and visible image fusion[J]. IEEE Sensors Journal, 2021, 21(13):14950-14959.
[30] Jian L, Rayhana R, Ma L, et al. Infrared and visible image fusion based on deep decomposition network and saliency analysis[J]. IEEE Transactions on Multimedia, 2021, 7(9):1-13.
[31] Ma C, Wang R, Zhou S, et al. The structural similarity index for IMRT quality assurance:radiomics-based error classification[J]. Medical Physics, 2021, 48(1):80-93.
[32] Zhang W, Wu Q J, Yang Y, et al. Fast ship detection with spatial-frequency analysis and ANOVA-based feature fusion[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19:1-5.
[33] Kanwal N, Jain S, Kaur P. Evaluating robustness for intensity based image registration measures using mutual information[J]. Cognitive Computing in Human Cognition:Perspectives and Applications, 2020, 17:73.