基于水印神经网络的可溯源DNN模型保护方法

doi:10.3969/j.issn.0255-8297.2023.02.001

摘要/Abstract

摘要： 针对深度神经网络（deep neural networks,DNN）模型安全与版权认证的问题，提出了一种多用户溯源的水印神经网络模型，通过密钥驱动生成水印图像，将其不可见地嵌入待保护目标模型的输出图像中，实现DNN模型的知识产权保护和版权追踪。在待保护的DNN模型中添加一种编解码器网络实现水印的嵌入，并使用双流篡改检测网络作为判别器，解决了模型的输出图像中可能出现的水印残留问题，提升了水印嵌入过程的不可感知性，减少了对DNN模型性能的影响，增强了安全性。此外，通过本文设计的双阶段训练法针对不同用户分发不同的含水印模型，当发生版权纠纷时，使用另一个残差网络可以从输出图像中提取水印图像。实验证明，本方法分发含水印的模型效率较高，并且即使对多个用户分发了嵌入相似水印图像的DNN模型，水印神经网络依然可以成功对模型进行溯源。

关键词: 深度神经网络, 数字水印, 版权保护, 水印神经网络, 图像隐写

Abstract: This paper proposes a multi-user traceability watermarking neural network approach to model security and copyright certification for deep neural networks (DNN). The watermark is generated by the key driver and embedded invisibly in the output images of the DNN model, hence realizing the intellectual property protection and copyright tracking of DNN model. A codec network is added to the DNN model to embed the watermark, and a two-stream tamper detection network is used as the discriminator. Thus, the problem of residual watermark in the output images of the model is solved, which, reduces the impact on the performance of DNN model and enhances the security. In addition, a two-stage training method is proposed in this paper to distribute different watermarked models to different users. When copyright disputes occur, another residual network can be used to extract the watermark image from the output image. Experiments show that the proposed method is efficient in distributing watermarked models, and is able to trace the source of DNN models embedded with similar watermarked images for multiple users.

Key words: deep neural networks, digital watermarking, intellectual property protection, watermarking neural networks, image steganography

中图分类号:

P391

刘雅蕾, 和红杰, 陈帆, 刘卓华. 基于水印神经网络的可溯源DNN模型保护方法[J]. 应用科学学报, 2023, 41(2): 183-196.

LIU Yalei, HE Hongjie, CHEN Fan, LIU Zhuohua. Traceable DNN Model Protection Based on Watermark Neural Network[J]. Journal of Applied Sciences, 2023, 41(2): 183-196.

参考文献

[1] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016:640-651.
[2] He K M, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:2980-2988.
[3] Zhu C C, Chen F Y, Ahmed U, et al. Semantic relation reasoning for shot-stable few-shot object detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021:8778-8787.
[4] Brock A, Donahue J, Simonyan K. Large scale GAN training for high fidelity natural image synthesis[C]//International Conference on Learning Representations, 2019.
[5] Zhang H, Koh J Y, Baldridge J, et al. Cross-modal contrastive learning for text-to-image generation[C]//2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021:833-842.
[6] Xue M F, Zhang Y S, Wang J, et al. Intellectual property protection for deep learning models:taxonomy, methods, attacks, and evaluations[J]. IEEE Transactions on Artificial Intelligence, 2022, 3(6):908-923.
[7] Tolosana R, Vera-Rodriguez R, Fierrez J, et al. Deepfakes and beyond:a survey of face manipulation and fake detection[J]. Information Fusion, 2020, 64:131-148.
[8] 张颖君, 陈恺, 周赓, 等. 神经网络水印技术研究进展[J]. 计算机研究与发展, 2021, 58(5):964-976. Zhang Y J, Chen K, Zhou G, et al. Research progress of neural networks watermarking technology[J]. Journal of Computer Research and Development, 2021, 58(5):964-976.(in Chinese)
[9] Uchida Y, Nagai Y, Sakazawa S, et al. Embedding watermarks into deep neural networks[C]//2017 ACM on International Conference on Multimedia Retrieval, 2017:269-277.
[10] Chen H L, Rouhani B D, Fu C, et al. DeepMarks:a secure fingerprinting framework for digital rights management of deep learning models[C]//2019 International Conference on Multimedia Retrieval, 2019:105-113.
[11] Rouhani B D, Chen H L, Koushanfar F. DeepSigns:an end-to-end watermarking framework for ownership protection of deep neural networks[C]//International Conference on Architectural Support for Programming Languages and Operating Systems, 2019:485-497.
[12] Wang J F, Wu H Z, Zhang X P, et al. Watermarking in deep neural networks via error back-propagation[J]. Electronic Imaging, 2020, 32(4):22-1-22-9.
[13] Guo J, Potkonjak M. Watermarking deep neural networks for embedded systems[C]//2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2018:1-8.
[14] Li Z, Hu C Y, Zhang Y, et al. How to prove your model belongs to you:a blind-watermark based framework to protect intellectual property of DNN[C]//35th Annual Computer Security Applications Conference, 2019:126-137.
[15] Shafieinejad M, Lukas N, Wang J Q, et al. On the robustness of backdoor-based watermarking in deep neural networks[C]//2021 ACM Workshop on Information Hiding and Multimedia Security, 2021:177-188.
[16] Wu H Z, Liu G, Yao Y W, et al. Watermarking neural networks with watermarked images[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(7):2591-2601.
[17] Zarrabi H, Emami A, Khadivi P, et al. BlessMark:a blind diagnostically-lossless watermarking framework for medical applications based on deep neural networks[J]. Multimedia Tools and Applications, 2020, 79(31):22473-22495.
[18] Zhang J, Chen D D, Liao J, et al. Deep model intellectual property protection via deep watermarking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8):4005-4020.
[19] Fan L X, Ng K W, Chan C S. Rethinking deep neural network ownership verification:embedding passports to defeat ambiguity attacks[DB/OL]. 2019[2021-06-08]. https://arxiv.org/abs/1909.07830.
[20] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11):139-144.
[21] Hu D H, Wang L, Jiang W J, et al. A novel image steganography method via deep convolutional generative adversarial networks[J]. IEEE Access, 2018, 6:38303-38314.
[22] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. 2015[2021-06-08]. https://arxiv.org/abs/1511.06434.
[23] 刘明明, 张敏情, 刘佳, 等. 基于生成对抗网络的无载体信息隐藏[J]. 应用科学学报, 2018, 36(2):371-382. Liu M M, Zhang M Q, Liu J, et al. Coverless information hiding based on generative adversarial networks[J]. Journal of Applied Sciences, 2018, 36(2):371-382. (in Chinese)
[24] Volkhonskiy D, Nazarov I, Burnaev E. Steganographic generative adversarial networks[C]//International Conference on Machine Vision (ICMV), 2020, 11433:991-1005.
[25] Shi H C, Dong J, Wang W, et al. SSGAN:secure steganography based on generative adversarial networks[C]//Pacific Rim Conference on Multimedia, 2018:534-544.
[26] Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C]//34th International Conference on Machine Learning, 2017:214-223.
[27] Qian Y L, Dong J, Wang W, et al. Deep learning for steganalysis via convolutional neural networks[C]//Media Watermarking, Security, and Forensics, 2015, 9409:171-180.
[28] Zhang C N, Benz P, Karjauv A, et al. UDH:universal deep hiding for steganography, watermarking, and light field messaging[C]//34th International Conference on Neural Information Processing Systems, 2020:10223-10234.
[29] Ronneberger O, Fischer P, Brox T. U-Net:convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and ComputerAssisted Intervention, 2015:234-241.
[30] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016:770-778.
[31] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:2242-2251.
[32] Zhou P, Han X T, Morariu V I, et al. Learning rich features for image manipulation detection[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018:1053-1061.
[33] Fridrich J, Kodovsky J. Rich models for steganalysis of digital images[J]. IEEE Transactions on information Forensics and Security, 2012, 7(3):868-882.
[34] Fan Q N, Yang J L, Hua G, et al. A generic deep architecture for single image reflection removal and image smoothing[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:3258-3267.
[35] Mirza M, Osindero S. Conditional generative adversarial nets[EB/OL]. 2014[2021-06-08]. https://arxiv.org/abs/1411.1784.