中文

Journal of Applied Sciences >

2023 , Vol. 41 >Issue 2: 183 - 196

DOI: https://doi.org/10.3969/j.issn.0255-8297.2023.02.001

Digital Media Forensics and Security

Traceable DNN Model Protection Based on Watermark Neural Network

Expand
  • 1. School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, Sichuan, China;
    2. School of Information Science and Technology, Southwest Jiaotong University, Chengdu 611756, Sichuan, China

Received date: 2021-06-08

  Online published: 2023-03-29

Fold

Abstract

This paper proposes a multi-user traceability watermarking neural network approach to model security and copyright certification for deep neural networks (DNN). The watermark is generated by the key driver and embedded invisibly in the output images of the DNN model, hence realizing the intellectual property protection and copyright tracking of DNN model. A codec network is added to the DNN model to embed the watermark, and a two-stream tamper detection network is used as the discriminator. Thus, the problem of residual watermark in the output images of the model is solved, which, reduces the impact on the performance of DNN model and enhances the security. In addition, a two-stage training method is proposed in this paper to distribute different watermarked models to different users. When copyright disputes occur, another residual network can be used to extract the watermark image from the output image. Experiments show that the proposed method is efficient in distributing watermarked models, and is able to trace the source of DNN models embedded with similar watermarked images for multiple users.

Key words： deep neural networks; digital watermarking; intellectual property protection; watermarking neural networks; image steganography

Cite this article

LIU Yalei, HE Hongjie, CHEN Fan, LIU Zhuohua . Traceable DNN Model Protection Based on Watermark Neural Network[J]. Journal of Applied Sciences, 2023 , 41(2) : 183 -196 . DOI: 10.3969/j.issn.0255-8297.2023.02.001

References

[1] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[C]//IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016:640-651.
[2] He K M, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:2980-2988.
[3] Zhu C C, Chen F Y, Ahmed U, et al. Semantic relation reasoning for shot-stable few-shot object detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021:8778-8787.
[4] Brock A, Donahue J, Simonyan K. Large scale GAN training for high fidelity natural image synthesis[C]//International Conference on Learning Representations, 2019.
[5] Zhang H, Koh J Y, Baldridge J, et al. Cross-modal contrastive learning for text-to-image generation[C]//2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021:833-842.
[6] Xue M F, Zhang Y S, Wang J, et al. Intellectual property protection for deep learning models:taxonomy, methods, attacks, and evaluations[J]. IEEE Transactions on Artificial Intelligence, 2022, 3(6):908-923.
[7] Tolosana R, Vera-Rodriguez R, Fierrez J, et al. Deepfakes and beyond:a survey of face manipulation and fake detection[J]. Information Fusion, 2020, 64:131-148.
[8] 张颖君, 陈恺, 周赓, 等. 神经网络水印技术研究进展[J]. 计算机研究与发展, 2021, 58(5):964-976. Zhang Y J, Chen K, Zhou G, et al. Research progress of neural networks watermarking technology[J]. Journal of Computer Research and Development, 2021, 58(5):964-976.(in Chinese)
[9] Uchida Y, Nagai Y, Sakazawa S, et al. Embedding watermarks into deep neural networks[C]//2017 ACM on International Conference on Multimedia Retrieval, 2017:269-277.
[10] Chen H L, Rouhani B D, Fu C, et al. DeepMarks:a secure fingerprinting framework for digital rights management of deep learning models[C]//2019 International Conference on Multimedia Retrieval, 2019:105-113.
[11] Rouhani B D, Chen H L, Koushanfar F. DeepSigns:an end-to-end watermarking framework for ownership protection of deep neural networks[C]//International Conference on Architectural Support for Programming Languages and Operating Systems, 2019:485-497.
[12] Wang J F, Wu H Z, Zhang X P, et al. Watermarking in deep neural networks via error back-propagation[J]. Electronic Imaging, 2020, 32(4):22-1-22-9.
[13] Guo J, Potkonjak M. Watermarking deep neural networks for embedded systems[C]//2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2018:1-8.
[14] Li Z, Hu C Y, Zhang Y, et al. How to prove your model belongs to you:a blind-watermark based framework to protect intellectual property of DNN[C]//35th Annual Computer Security Applications Conference, 2019:126-137.
[15] Shafieinejad M, Lukas N, Wang J Q, et al. On the robustness of backdoor-based watermarking in deep neural networks[C]//2021 ACM Workshop on Information Hiding and Multimedia Security, 2021:177-188.
[16] Wu H Z, Liu G, Yao Y W, et al. Watermarking neural networks with watermarked images[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(7):2591-2601.
[17] Zarrabi H, Emami A, Khadivi P, et al. BlessMark:a blind diagnostically-lossless watermarking framework for medical applications based on deep neural networks[J]. Multimedia Tools and Applications, 2020, 79(31):22473-22495.
[18] Zhang J, Chen D D, Liao J, et al. Deep model intellectual property protection via deep watermarking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8):4005-4020.
[19] Fan L X, Ng K W, Chan C S. Rethinking deep neural network ownership verification:embedding passports to defeat ambiguity attacks[DB/OL]. 2019[2021-06-08]. https://arxiv.org/abs/1909.07830.
[20] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11):139-144.
[21] Hu D H, Wang L, Jiang W J, et al. A novel image steganography method via deep convolutional generative adversarial networks[J]. IEEE Access, 2018, 6:38303-38314.
[22] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. 2015[2021-06-08]. https://arxiv.org/abs/1511.06434.
[23] 刘明明, 张敏情, 刘佳, 等. 基于生成对抗网络的无载体信息隐藏[J]. 应用科学学报, 2018, 36(2):371-382. Liu M M, Zhang M Q, Liu J, et al. Coverless information hiding based on generative adversarial networks[J]. Journal of Applied Sciences, 2018, 36(2):371-382. (in Chinese)
[24] Volkhonskiy D, Nazarov I, Burnaev E. Steganographic generative adversarial networks[C]//International Conference on Machine Vision (ICMV), 2020, 11433:991-1005.
[25] Shi H C, Dong J, Wang W, et al. SSGAN:secure steganography based on generative adversarial networks[C]//Pacific Rim Conference on Multimedia, 2018:534-544.
[26] Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C]//34th International Conference on Machine Learning, 2017:214-223.
[27] Qian Y L, Dong J, Wang W, et al. Deep learning for steganalysis via convolutional neural networks[C]//Media Watermarking, Security, and Forensics, 2015, 9409:171-180.
[28] Zhang C N, Benz P, Karjauv A, et al. UDH:universal deep hiding for steganography, watermarking, and light field messaging[C]//34th International Conference on Neural Information Processing Systems, 2020:10223-10234.
[29] Ronneberger O, Fischer P, Brox T. U-Net:convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and ComputerAssisted Intervention, 2015:234-241.
[30] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016:770-778.
[31] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:2242-2251.
[32] Zhou P, Han X T, Morariu V I, et al. Learning rich features for image manipulation detection[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018:1053-1061.
[33] Fridrich J, Kodovsky J. Rich models for steganalysis of digital images[J]. IEEE Transactions on information Forensics and Security, 2012, 7(3):868-882.
[34] Fan Q N, Yang J L, Hua G, et al. A generic deep architecture for single image reflection removal and image smoothing[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017:3258-3267.
[35] Mirza M, Osindero S. Conditional generative adversarial nets[EB/OL]. 2014[2021-06-08]. https://arxiv.org/abs/1411.1784.
Options
Outlines

/

〈 〉