A low light image enhancement method based on color restoration transformer networks (CRTNet) is proposed to address the issue of image and color distortion in low light environments. This method combines channel attention and spatial attention mechanisms. CRTNet consists of a color attention module (CAM), a color map module (CMM), and a sequential enhancement structure. Firstly, CAM is divided into two parts: color channel attention module and color space attention module. Utilizing the global information capture capability of Transformer, the color channel attention module emphasizes meaningful color channels by assigning higher weights to generate channel attention vectors. The color space attention module uses a three-layer convolution structure, focuses on spatial details in high-dimensional space and generates spatial attention weight map. Secondly, CMM extracts high-dimensional image features through a linear fitting process, scaling and shifting these features in the 64D space across both channel and spatial dimensions to obtain global and detail image information. By combining with the original image features, it supplements the color, brightness, contrast, and detail information in the original image features to achieve color enhancement. Finally, a sequential enhancement structure is adopted to repeat CAM and CMM operations three times with the output of CMM serving as input, in order to fit higher-order function mappings and effectively enhance low light images. Experiments results and user studies on public datasets demonstrate that the proposed method outperforms existing approaches in quantitative measurement, detail and color restoration.
JIANG Zetao, HUANG Jingfan, ZHU Wencai, HUANG Qinyang, JIN Xin
. A Low Light Image Enhancement Method Based on CRTNet[J]. Journal of Applied Sciences, 2024
, 42(6)
: 934
-946
.
DOI: 10.3969/j.issn.0255-8297.2024.06.004
[1] Cheng H D, Shi X J. A simple and effective histogram equalization approach to image enhancement [J]. Digital Signal Processing, 2004, 14(2): 158-170.
[2] Jobson D J, Rahman Z, Woodell G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes [J]. IEEE Transactions on Image Processing, 1997, 6(7): 965-976.
[3] Guo X J, Li Y, Ling H B. LIME: low-light image enhancement via illumination map estimation [J]. IEEE Transactions on Image Processing, 2017, 26(2): 982-993.
[4] Zhao Z J, Xiong B S, Wang L, et al. RetinexDIP: a unified deep framework for low-light image enhancement [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(3): 1076-1088.
[5] Liu R S, Ma L, Zhang J A, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021: 10556-10565.
[6] Wu W H, Weng J, Zhang P P, et al. URetinex-net: retinex-based deep unfolding network for low-light image enhancement [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022: 5891-5900.
[7] Feng W, Wu G M, Zhou S Q, et al. Low-light image enhancement based on Retinex-Net with color restoration [J]. Applied Optics, 2023, 62(25): 6577-6584.
[8] Dong X, Pang Y A, Wen J G. Fast efficient algorithm for enhancement of low lighting video [DB/OL]. 2010[2023-11-15]. https://dl.acm.org/doi/10.1145/1836845.1836920.
[9] Park D, Kim M, Ku B, et al. Image enhancement for extremely low light conditions [C]//11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2014: 307-312.
[10] Yang Y, Wang C Y, Liu R S, et al. Self-augmented unpaired image dehazing via density and depth decomposition [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022: 2027-2036.
[11] Jiang Y F, Gong X Y, Liu D, et al. EnlightenGAN: deep light enhancement without paired supervision [J]. IEEE Transactions on Image Processing, 2021, 30: 2340-2349.
[12] Zhang Y, Di X G, Zhang B, et al. Self-supervised low light image enhancement and denoising [DB/OL]. 2021[2023-11-15]. http://arxiv.org/abs/2103.00832.
[13] Wang Y F, Wan R J, Yang W H, et al. Low-light image enhancement with normalizing flow [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(3): 2604-2612.
[14] Liang D, Li L, Wei M Q, et al. Semantically contrastive learning for low-light image enhancement [J]. AAAI Conference on Artificial Intelligence, 2022, 36(2): 1555-1563.
[15] Bychkovsky V, Paris S, Chan E, et al. Learning photographic global tonal adjustment with a database of input/output image pairs [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2011: 97-104.
[16] Zeng H, Cai J R, Li L D, et al. Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time [J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(4): 2058-2073.
[17] He J W, Liu Y H, Qiao Y, et al. Conditional sequential modulation for efficient global image retouching [C]//European Conference on Computer Vision, 2020: 679-695.
[18] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [C]// 31st International Conference on Neural Information Processing Systems. Red Hook, 2017: 6000-6010.
[19] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale [DB/OL]. 2020[2023-11-15]. http://arxiv.org/abs/2010.11929.
[20] Cui Z T, Li K C, Gu L, et al. You only need 90K parameters to adapt light: a light weight transformer for image enhancement and exposure correction [DB/OL]. 2022[2023-11- 15]. http://arxiv.org/abs/2205.14871.
[21] Cai Y H, Bian H, Lin J, et al. Retinexformer: one-stage retinex-based transformer for low-light image enhancement [DB/OL]. 2023[2023-11-15]. http://arxiv.org/abs/2303.06705.
[22] Guo C L, Li C Y, Guo J C, et al. Zero-reference deep curve estimation for low-light image enhancement [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 1777-1786.
[23] Ma L, Ma T Y, Liu R S, et al. Toward fast, flexible, and robust low-light image enhancement [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022: 5627-5636.
[24] Hu J, Shen L, Sun G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 7132-7141.
[25] Jaderberg M, Simonyan K, Zisserman A, et al. Spatial transformer networks [DB/OL]. 2015[2023-11-15]. http://arxiv.org/abs/1506.02025.
[26] Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module [C]//European Conference on Computer Vision, 2018: 3-19.
[27] Hu Y M, He H, Xu C X, et al. Exposure: a white-box photo post-processing framework [DB/OL]. 2017[2023-11-15]. http://arxiv.org/abs/1709.09602.
[28] Hwang S J, Kapoor A, Kang S B. Context-based automatic local image enhancement [C]//European Conference on Computer Vision, 2012: 569-582.
