多层记忆增强生成对抗网络二次预测的视频异常检测方法

doi:10.3969/j.issn.0255-8297.2023.01.007

Abstract

Abstract: In order to improve the accuracy of video anomaly detection, we propose a video anomaly detection method based on secondary prediction of multi-layer memory enhancement generative adversarial networks. Firstly, a spatiotemporal cube is extracted from target detection, and sent into encoder to obtain a prediction frame. Secondly, the apparent feature of the prediction frame and the optical flow feature of corresponding real frames are fused to form fusion features. Finally, a secondary prediction future frame is generated by using multi-layer memory enhancement generative adversarial networks, for learning normal feature patterns of different levels and capturing the semantic information of context. Experimental results on UCSD Ped2 and CUHK Avenue datasets show that the proposed method can effectively improve the performance of video anomaly detection compared with other video anomaly detection methods, and its frame level AUC reaches 99.57% and 91.59%, respectively.

Key words: video anomaly detection, multi-layer memory enhancement, generative adversarial network, future frame prediction, deep learning

CLC Number:

TP391

ZENG Jing, LI Ying, QI Xiaosha, JI Genlin. Video Anomaly Detection Method Based on Secondary Prediction of Multi-layer Memory Enhancement Generative Adversarial Network[J]. Journal of Applied Sciences, 2023, 41(1): 80-94.

References

[1] 杨帆, 肖斌, 於志文. 监控视频的异常检测与建模综述[J]. 计算机研究与发展, 2021, 58(12):2708-2723. Yang F, Xiao B, Yu Z W. Anomaly detection and modeling of surveillance video[J]. Journal of Computer Research and Development, 2021, 58(12):2708-2723. (in Chinese)
[2] Liu W, Luo W X, Lian D Z, et al. Future frame prediction for anomaly detection-a new baseline[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018:6536-6545.
[3] Lee S, Kim H G, Ro Y M. BMAN:bidirectional multi-scale aggregation networks for abnormal event detection[J]. IEEE Transactions on Image Processing, 2020, 29:2395-2408.
[4] 冷佳旭, 谭明圮, 胡波, 等. 基于隐式视角转换的视频异常检测[J]. 计算机科学, 2022, 49(2):142-148. Leng J X, Tan M P, Hu B, et al. Video anomaly detection bases on implicit view transformation[J]. Computer Science, 2022, 49(2):142-148. (in Chinese)
[5] Sabokrou M, Fayyaz M, Fathy M, et al. Deep-cascade:cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Image Processing, 2017, 26(4):1992-2004.
[6] 江帆, 王贵锦, 林行刚, 等. 基于运动方向的异常行为检测[J]. 自动化学报, 2008, 34(11):1348-1357. Jiang F, Wang G J, Lin X G, et al. Anomaly detection based on motion direction[J]. Acta Automatica Sinica, 2008, 34(11):1348-1357. (in Chinese)
[7] 李欣璐, 吉根林, 赵斌. 基于卷积自编码器分块学习的视频异常事件检测与定位[J]. 数据采集与处理, 2021, 36(3):489-497. Li X L, Ji G L, Zhao B. Convolutional auto-encoder patch learning based video anomaly event detection and localization[J]. Journal of Data Acquisition and Processing, 2021, 36(3):489-497. (in Chinese)
[8] 魏明军, 周太宇, 纪占林, 等. 基于Mask-YOLO的复杂场景口罩佩戴检测[J]. 应用科学学报, 2022, 40(1):93-104. Wei M J, Zhou T Y, Ji Z L, et al. Mask wearing detection in complex scenes based on Mask-YOLO[J]. Journal of Applied Sciences, 2022, 40(1):93-104. (in Chinese)
[9] 汪鹏, 郑文凤, 史进, 等. 基于MFANet和上下文特征融合的遥感影像目标检测[J]. 应用科学学报, 2022, 40(1):131-144. Wang P, Zheng W F, Shi J, et al. Remote sensing image object detection based on MFANet and contextual features fusion[J]. Journal of Applied Sciences, 2022, 40(1):131-144. (in Chinese)
[10] Ionescu R T, Khan F S, Georgescu M I, et al. Object-centric auto-encoders and dummy anomalies for abnormal event detection in video[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 2019:7842-7851.
[11] Yu G, Wang S Q, Cai Z P, et al. Cloze test helps:effective video anomaly detection via learning to complete video events[C]//Proceedings of the 28th ACM International Conference on Multimedia, Seattle, 2020:583-591.
[12] Cai Z W, Vasconcelos N. Cascade R-CNN:delving into high quality object detection[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018:6154-6162.
[13] Gong D, Liu L Q, Le V, et al. Memorizing normality to detect anomaly:memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul, 2019:1705-1714.
[14] Park H, Noh J, Ham B. Learning memory-guided normality for anomaly detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020:14360-14369.
[15] Ilg E, Mayer N, Saikia T, et al. FlowNet 2.0:evolution of optical flow estimation with deep networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, 2017:1647-1655.
[16] Hasan M, Choi J, Neumann J, et al. Learning temporal regularity in video sequences[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016:733-742.
[17] Zhao Y R, Deng B, Shen C, et al. Spatio-temporal autoencoder for video anomaly detection[C]//Proceedings of 2017 ACM on Multimedia Conference, Mountain View, 2017:1933-1941.
[18] Tang Y, Zhao L, Zhang S S, et al. Integrating prediction and reconstruction for anomaly detection[J]. Pattern Recognition Letters, 2020, 129:123-130.
[19] Nguyen T N, Meunier J. Anomaly detection in video sequence with appearance-motion correspondence[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision, Seoul, 2019:1273-1283.
[20] Ye M C, Peng X J, Gan W H, et al. AnoPCN:video anomaly detection via deep predictive coding network[C]//Proceedings of the 27th ACM International Conference on Multimedia, 2019:1805-1813.
[21] Luo W X, Liu W, Lian D Z, et al. Video anomaly detection with sparse coding inspired deep neural networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(3):1070-1084.
[22] Cai R C, Zhang H, Liu W, et al. Appearance-motion memory consistency network for video anomaly detection[C]//Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, Virtual, 2021:938-946.
[23] Lv H, Chen C, Cui Z, et al. Learning normal dynamics in videos with meta prototype network[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Virtual, 2021:15425-15434.
[24] Fan Y X, Wen G J, Li D, et al. Video anomaly detection and localizati6on via Gaussian mixture fully convolutional variational autoencoder[J]. Computer Vision and Image Understanding, 2020, 195:102920.
[25] Li N J, Chang F L, Liu C S. Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes[J]. IEEE Transactions on Multimedia, 2021, 23:203-215.
[26] Doshi K, Yilmaz Y. Continual learning for anomaly detection in surveillance videos[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020:1025-1034.

Video Anomaly Detection Method Based on Secondary Prediction of Multi-layer Memory Enhancement Generative Adversarial Network

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	QIN Jing, HAN Yue, WANG Liyong, JI Changqing, LIU Lu, WANG Zumin. An Automatic Atrial Fibrillation Detection Model Based on GAN and MS-ResNet [J]. Journal of Applied Sciences, 2024, 42(1): 15-26.
[2]	SHEN Kunye, ZHOU Xiaofei, FEI Xiaobo, CHEN Yuzhong, ZHANG Jiyong, YAN Chenggang. Boundary-Aware Deeply Residual Network for Salient Object Detection of Strip Steel Surface Defects [J]. Journal of Applied Sciences, 2023, 41(6): 978-988.
[3]	ZHANG Chunsen, ZHU Jiangle, ZHANG Xuefen, LIU Xudong, SHI Shu. Void Filling of DEM in a Generative Adversarial Network Fused with Self-Attention Mechanism [J]. Journal of Applied Sciences, 2023, 41(5): 789-800.
[4]	HUANG Xianpei, MENG Qingxiang. Land Cover Classification of Sentinel-2 Image Based on Multi-feature Convolution Neural Network [J]. Journal of Applied Sciences, 2023, 41(5): 766-776.
[5]	WEI Jieling, MA Xiuli, JIN Yanliang, WANG Rui. Encrypted Traffic Classification Algorithm Based on VPN Channel [J]. Journal of Applied Sciences, 2023, 41(4): 646-656.
[6]	XIE Yiyi, ZHANG Yushu, ZHAO Ruoyu, WEN Wenying, ZHOU Yuqian. Image Privacy Protection Based on Cycle-Consistent Generative Adversarial Networks [J]. Journal of Applied Sciences, 2023, 41(2): 228-239.
[7]	XIAO Xiaotong, DING Jianwei, ZHANG Qi. Segmented Backdoor Defense Based on Local Gradient and Global Gradient Ascent [J]. Journal of Applied Sciences, 2023, 41(2): 218-227.
[8]	WANG Ting, WANG Na, CUI Yunpeng, LIU Juan. Medical Electronic Data Feature Learning Method Based on Deep Learning [J]. Journal of Applied Sciences, 2023, 41(1): 41-54.
[9]	NIE Jianghua, XIAO Yongsheng, HUANG Lizhen, HE Fengshou. High Resolution Range Profile Radar Target Recognition Based on Time-Frequency Analysis and Deep Learning [J]. Journal of Applied Sciences, 2022, 40(6): 973-983.
[10]	JIANG Xiaoyong, LI Zhongyi, HUANG Langyue, PENG Mengle, XU Shuyang. Review of Neural Network Pruning Techniques [J]. Journal of Applied Sciences, 2022, 40(5): 838-849.
[11]	LUO Changyin, CHEN Xuebin, SONG Shangwen, ZHANG Shufen, LIU Zhiyu. Federated Ensemble Algorithm Based on Deep Neural Network [J]. Journal of Applied Sciences, 2022, 40(3): 493-510.
[12]	LEI Qianhui, PAN Lili, SHAO Weizhi, HU Haipeng, HUANG Yao. Segmentation Model of COVID-19 Lesions Based on Triple Attention Mechanism [J]. Journal of Applied Sciences, 2022, 40(1): 105-115.
[13]	ZHU Li, YANG Qing, WU Tao, LI Chen, LI Ming. Emotional Analysis of Brain Waves Based on CNN and Bi-LSTM [J]. Journal of Applied Sciences, 2022, 40(1): 1-12.
[14]	OU Qiaofeng, XIAO Jiabing, XIE Qunqun, XIONG Bangshu. Multi-target Detection and Recognition for Vehicle Inspection Images Based on Deep Learning [J]. Journal of Applied Sciences, 2021, 39(6): 939-951.
[15]	KANG Huixian, YI Biao, WU Hanzhou. Recent Advances in Text Steganography and Steganalysis [J]. Journal of Applied Sciences, 2021, 39(6): 923-938.