光场（light field,LF）成像同时记录光线的位置和角度信息，这种高维特性导致光场数据量急剧增加。因此，高效的光场压缩技术成为该领域研究和开发的重点。近年来，学术界提出了多种基于深度学习的光场压缩方法。然而，这些方法往往难以实现端到端的联合优化，并且需要显式传递视差或几何信息，从而显著增加了编码方案的复杂性。为了解决该问题，本文提出一种全新的端到端光场压缩模型。该模型基于光场视点之间的视差关系，采用可变形注意力机制进行视差补偿，通过对视差特征和残差的编解码实现光场数据的有效压缩。实验结果表明，本文提出的方法在率失真性能上优于其他现有光场压缩方法，且在中高码率编码性能上达到了最先进水平。
Light field(LF) imaging captures both the positional and angular information of light rays, leading to a significant increase in LF data volume due to its high-dimensional characteristics. As a result, efficient LF compression techniques have become an important research focus in this field. In recent years, researchers have proposed various deep learningbased methods for LF compression. However, these methods often struggle to achieve endto-end joint optimization and require the explicit transmission of disparity or geometric information, which significantly increases the complexity of the coding scheme. To address this issue, this paper proposed a novel end-to-end LF compression model. The model utilized disparity relationships among LF views and used a deformable attention mechanism for disparity compensation, enabling effective LF compression by encoding and decoding disparity features and residuals. Experimental results show that the proposed method outperforms other LF compression methods in rate-distortion performance and achieves state-of-the-art performance in mid-to-high bitrate coding.
[1] Levoy M, Hanrahan P. Light field rendering [C]//23rd Annual Conference on Computer Graphics and Interactive Technique, 1996: 31-42.
[2] 张驰, 刘菲, 侯广琦, 等. 光场成像技术及其在计算机视觉中的应用[J]． 中国图象图形学报, 2016, 21(3): 263-281． Zhang C, Liu F, Hou G Q, et al. Light field photography and its application in computer vision [J]. Journal of Image and Graphics, 2016, 21(3): 263-281. (in Chinese)
[3] Han K, Xiang W, Wang E, et al. A novel occlusion-aware vote cost for light field depth estimation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(11): 8022-8035.
[4] Galea C, Guillemot C. Denoising of 3D point clouds constructed from light fields [C]//44th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019: 1882-1886.
[5] Lv X Q, Wang X, Wang Q, et al. 4D light field segmentation from light field super-pixel hypergraph representation [J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(9): 3597-3610.
[6] Zhang Q D, Wang S Q, Wang X, et al. A multi-task collaborative network for light field salient object detection [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(5): 1849-1861.
[7] 刘宇洋, 朱策, 郭红伟． 光场数据压缩研究综述[J]． 中国图象图形学报， 2019， 24(11)： 1842-1859. Liu Y Y, Zhu C, Guo H W. Survey of light field data compression [J]. Journal of Image and Graphics, 2019, 24(11): 1842-1859. (in Chinese)
[8] Conti C, Soares L D, Nunes P. Dense light field coding: a survey [J]. IEEE Access, 2020, 8: 49244-49284.
[9] Jiang X R, Le Pendu M, Guillemot C. Light field compression using depth image based view synthesis [C]//IEEE International Conference on Multimedia and Expo (ICME), 2017: 19-24.
[10] Viola I, Maretic H P, Frossard P, et al. A graph learning approach for light field image compression [C]//Conference on Applications of Digital Image Processing XLI, 2018: 126-137.
[11] Liu D Y, Huang Y, Fang Y M, et al. Multi-stream dense view reconstruction network for light field image compression [J]. IEEE Transactions on Multimedia, 2023, 25: 4400-4414.
[12] Cheng Z X, Sun H M, Takeuchi M, et al. Learned image compression with discretized Gaussian mixture likelihoods and attention modules [C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 7939-7948.
[13] Dai J F, Qi H Z, Xiong Y W, et al. Deformable convolutional networks [C]//16th IEEE International Conference on Computer Vision (ICCV), 2017: 764-773.
[14] Ballé J, Minnen D, Singh S, et al. Variational image compression with a scale hyperprior [DB/OL]. (2018-05-01) [2024-05-17]. https://arxiv.org/abs/1802.01436.
[15] Lin K, Jia C M, Zhang X F, et al. DMVC: decomposed motion modeling for learned video compression [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(7): 3502-3515.
[16] Rerabek M, Ebrahimi T. New light field image dataset [EB/OL]. [2024-05-17]. https://infoscience.epfl.ch/entities/publication/75ea92d6-332a-4ff0-bfa5-0eb0d8bbcd5c.
[17] Bjontegaard G. Calculation of average PSNR differences between RD-curves [EB/OL]. [2024- 05-17]. https://www.itu.int/wftp3/av-arch/video-site/0104_Aus/VCEG-M33.doc.
[18] Chen K, Wang J Q, Pang J M, et al. MMDetection: open MMlab detection toolbox and benchmark [DB/OL]. (2019-06-17) [2024-05-17]. https://arxiv.org/abs/1906.07155.
[19] Liu D, Wang L Z, Li L, et al. Pseudo-sequence-based light field image compression [C]//IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2016: 1-4.
[20] Liu D Y, An P, Ma R, et al. Content-based light field image compression method with Gaussian process regression [J]. IEEE Transactions on Multimedia, 2020, 22(4): 846-859.
[21] Sullivan G J, Ohm J R, Han W J, et al. Overview of the high efficiency video coding (HEVC) standard [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(12): 1649-1668.
[22] De Carvalho M B, Pereira M P, Pagliari C L, et al. A 4D DCT-based lenslet light field codec [C]//25th IEEE International Conference on Image Processing (ICIP), 2018: 435-439.
[23] Schelkens P, Astola P, Da Silva E A B, et al. JPEG Pleno light field coding technologies [C]//Conference on Applications of Digital Image Processing XLII, 2019: 391-401.
