基于DTS-ILDA模型和关联过滤的新闻话题演化分析

doi:10.3969/j.issn.0255-8297.2017.05.009

Abstract

Abstract:

In topic evolution and tracking, as the size of time slices and the K value of the topic model are fxed, it is hard to locate important time turning points, which is prone to error topic correlation in the evolutionary analysis. To solve the problem, we propose an improved dynamic temporal segmentation-infnite latent Dirichlet allocation (DTS-ILDA) model and an associated fltering mechanism. The model combines an improved dynamic time segmentation algorithm with an infnite latent Dirichlet allocation (ILDA) model to extract topics. Dynamic time segmentation algorithm traverses the data set according to the time sequence, and then uses a contingency table to analysis the distribution of topics to measure the segmentation results and an ILDA model to extract K topics. In addition, an association fltering mechanism is proposed for error prone association in the evolutionary analysis. It removes weak association relationship. Finally, fve evolutionary relationships of right subtopic association are established according to the time sequence relationship. Experiments show that the presented method can effectively fnd important time points when the main content of the topic changes, preventing generation of meaningless topics. It can also reduce error-topic related interference, extracting exact deep relationship between the topics.

Key words: topic model, topic evolution, a fltering, temporal segment, infnite latent Dirichlet allocation (ILDA) mode

CLC Number:

TP391

GUO Xiao-li, ZHOU Zi-lan, LIU Yao-wei, DU Jian-hong, HUANG Yan. Analysis of News Topic Evolution Based on DTS-ILDA Model and Association Filtering[J]. Journal of Applied Sciences, 2017, 35(5): 634-646.

References

[1] Cui W W, Liu S X, Tan L, Shi C L, Song Y Q, Gao Z J, Tong X, Qu H M. TextFlow:towards better understanding of evolving topics in text[J]. IEEE Transactions on Visualization & Computer Graphics, 2011, 17(12):2412-21.
[2] 曲朝阳,范旭东,曲楠,于华涛. 基于本体的智能电网文本知识获取模型[J]. 东北电力大学学报,2014, 34(5):60-68. Qu Z Y, Fan X D, Qu N, Yu H T. Smart grid text knowledge acquisition model based on ontology[J]. Journal of Northeast Dianli University, 2014, 34(5):60-68. (in Chinese)
[3] 曹丽娜,唐锡晋. 基于主题模型的BBS话题演化趋势分析[J]. 管理科学学报,2014, 17(11):109-121. Cao L N, Tang X J. Trends of BBS topics based on dynamic topic model[J]. Journal of Mangement Sciences in China, 2014, 17(11):109-121. (in Chinese)
[4] 洪宇,仓玉,朱巧明,姚建民,周国栋. 话题跟踪中静态和动态话题模型的核捕捉衰减[J]. 软件学报,2012, 23(5):1100-1119. Hong Y, Cang Y, Zhu Q M, Yao J M, Zhou G D. Descending kernel track of static and dynamic topic models in topic tracking[J]. Journal of Software, 2012, 23(5):1100-1119. (in Chinese)
[5] 徐戈,王厚峰. 自然语言处理中主题模型的发展[J]. 计算机学报,2011, 34(8):1423-1436. Xu G, Wang H F. The development of topic models in natural language processing[J]. Chinese Journal of Computers, 2011, 34(8):1423-1436. (in Chinese)
[6] 郭晓利,韩啸. 电网知识协同发现策略研究[J]. 东北电力大学学报,2014, 34(1):94-98. Guo X L, Han X. Grid knowledge collaborative discovery strategy research[J]. Journal of Northeast Dianli University, 2014, 34(1):94-98. (in Chinese)
[7] 杨玉珍,刘培玉,费绍栋,张成功. 融合扩展信息瓶颈理论的话题关联检测方法研究[J]. 自动化学报,2014, 40(3):471-479. Yang Y Z, Liu P Y, Fei S D, Zhang C G. A topic link detection method based on improved information bottleneck theory[J]. Acta Automatica Sinica, 2014, 40(3):471-479. (in Chinese)
[8] Hospedales T, Gong S, Xiang T. Video behavior mining using a dynamic topic model[J]. International Journal of Computer Vision, 2012, 98(3):303-323.
[9] Alsumait L, Barbar Daniel, Domeniconi C. On-line LDA:adaptive topic models for mining text streams with applications to topic detection and tracking[C]//IEEE International Conference on Data Mining. Pisa:IEEE, 2008:3-12.
[10] 胡艳丽,白亮,张维明. 一种话题演化建模与分析方法[J]. 自动化学报,2012, 38(10):1690-1697. Hu Y L, Bai L, Zhang W M. Modeling and analyzing topic evolution[J]. Acta Automatica Sinica, 2012, 38(10):1690-1697. (in Chinese)
[11] Wang X, Mccallum A. Topics over time:a non-Markov continuous-time model of topical trends[C]//ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Philadelphia:ACM, 2006:424-433.
[12] Hall D, Jurafsky D, Manning C D. Studying the history of ideas using topic models[C]//Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. PA:ACM, 2008:363-371.
[13] 赵旭剑,李波,杨春明,张晖,金培权,岳丽华,戴文锴. 一种基于特征演变的新闻话题演化挖掘方法[J]. 计算机学报,2014, 37(4):819-832. Zhao X J, Li B, Yang C M, Zhang H, Jing P Q, Yue L H, Dai W K. A topic evolution mining algorithm of news text based on feature evolving[J]. Chinese Journal of Computers, 2014, 37(4):819-832. (in Chinese)
[14] Brody S, Elhadad N. An unsupervised aspect-sentiment model for online reviews[C]//Human Language Technologies:Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, Los Angeles, California, USA. DBLP, 2013:804-812.
[15] Pan S M, Zhou M, Song Y Q, Qian W H, Wang F, Liu S X. Optimizing temporal topic segmentation for intelligent text visualization[C]//International Conference on Intelligent User Interfaces. Santa Monica:ACM, 2013:348-353.
[16] Gao Z, Song Y, Liu S, Wang H, Wei H, Chen Y, Cui W. Tracking and connecting topics via incremental hierarchical Dirichlet processes[C]//2011 IEEE 11th International Conference on Data Mining. Vancouver:IEEE, 2011:1056-1061.
[17] Gad S, Ramakrishnan N, Hampton K N, Kavanaugh A. Bridging the divide in democratic engagement:studying conversation patterns in advantaged and disadvantaged communities[C]//International Conference on Social Informatics. Alexandria:IEEE, 2012:165-176.
[18] 吕楠,罗军勇,刘尧,杨慧洁. 一种有效的事件演化分析算法[J]. 计算机应用研究,2009, 26(11):4101-4103. Lü N, Luo J Y, Liu Y, Yang H J. Effective event evolution analysis algorithm[J]. Application Research of Computers, 2009, 26(11):4101-4103. (in Chinese)
[19] 胡艳丽,白亮,张维明. 网络舆情中一种基于OLDA的在线话题演化方法[J]. 国防科技大学学报,2012, 34(1):150-154. Hu Y L, Bai L, Zhang W M. OLDA-based method for online topic evolution in network public opinion analysis[J]. Journal of National University of Defense Technology, 2012, 34(1):150-154. (in Chinese)
[20] Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation[J]. Journal of Machine Learning Research, 2003, 3:993-1022.
[21] Heinrich G. "Infnite LDA"-implementing the HDP with minimum code complexity[J]. Technical Note, 2011, 20(1):114-134.
[22] Teh Y W, Blei D M. Hierarchical Dirichlet processes[J]. Journal of the American Statistical Association, 2004, 101(467):1566-1581.
[23] Ding W, Chen C. Dynamic topic detection and tracking:a comparison of HDP, C-word, and cocitation methods[J]. Journal of the Association for Information Science & Technology, 2014, 65(10):2084-2097.
[24] 赵凡. 基于共词分析的学科主题动态跟踪相似算法改进研究[J]. 情报杂志,2010, 29(1):173-176. Zhao F. Research on similarity algorithm improvement of dynamic tracing disciplinary themes based on Co-word analysis[J]. Journal of Intelligence,2010, 29(1):173-176. (in Chinese)
[25] 李保利,杨星. 基于LDA模型和话题过滤的研究主题演化分析[J]. 小型微型计算机系,2012, 33(12):2738-2743. Li B L, Yang X. Analyzing research topic evolution with LDA and topic fltering[J]. Journal of Chinese Computer Systems, 2012, 33(12):2738-2743. (in Chinese)
[26] 曲朝阳,陈帅,杨帆,朱莉. 基于云计算技术的电力大数据预处理属性约简方法[J]. 电力系统自动化,2014, 38(8):67-71. Qu Z Y, Chen S, Yang F, Zhu L. An attribute reducing method for electric power big data preprocessing based on cloud computing technology[J]. Automation of Electric Power Systems, 2014, 38(8):67-71. (in Chinese)

Analysis of News Topic Evolution Based on DTS-ILDA Model and Association Filtering

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	YAO Rujing, YANG Lei, YANG Tao, HU Yingxin, TIAN Qiang, WU Ou. Analysis for Psychological Scale Big Data Based on Improved Ising Model [J]. Journal of Applied Sciences, 2020, 38(3): 339-352.
[2]	JIANG Qinyin, ZHANG Xi. Topic-Specific Assessment Approach for Social Network Influence Evaluation [J]. Journal of Applied Sciences, 2020, 38(3): 353-366.
[3]	ZHANG Tongtong, DONG Junyu, ZHAO Haoran, LI Qiong, SUN Xin. Lightweight Phytoplankton Detection Network Based on Knowledge Distillation [J]. Journal of Applied Sciences, 2020, 38(3): 367-376.
[4]	WEI Ziyang, ZHAO Zhihong, ZHAO Jingjiao. Improved Faster R-CNN Algorithm and Its Application on Vehicle Detection [J]. Journal of Applied Sciences, 2020, 38(3): 377-387.
[5]	SUN Ronghai, SHI Linfu, YU Chunqiang, LAO Huan, TANG Zhenjun. Random Interpolation Method for Data Hiding in Encrypted Images [J]. Journal of Applied Sciences, 2020, 38(3): 419-430.
[6]	WANG Daqian, CUI Rongyi, JIN Jingxuan. Text Detection in Natural Scene Based on Visual Attention Model and Multi-scale MSER [J]. Journal of Applied Sciences, 2020, 38(3): 496-506.
[7]	YANG Gaoqiang, LI Sheng, ZHANG Xinpeng. Altered Fingerprint Detection Based on Orientation Field [J]. Journal of Applied Sciences, 2019, 37(6): 775-782.
[8]	YU Shuzhen, OU Qiaofeng, XIONG Bangshu, CHEN Yaofeng. A Calibration Method Based on Defocused Image for Stereo Vision Sensor with Large FOV [J]. Journal of Applied Sciences, 2019, 37(6): 795-805.
[9]	CUI Sanshuai, MAO Maoyu, LIN Xiaodan, KANG Xiangui. Survey on Analysis and Applications of Electric Network Frequency (ENF) Signals in Image and Video [J]. Journal of Applied Sciences, 2019, 37(5): 573-589.
[10]	LIU Wei, ZHANG Wanling, XIANG Shijun. Face Anti-spoofing Based on LBP, Multilayer DCT and CNN [J]. Journal of Applied Sciences, 2019, 37(5): 609-617.
[11]	QU Lingfeng, CHEN fan, HE Hongjie, YUAN yuan. Security Analysis of Image Encryption Algorithm Based on Bit Plane-Pixel Block Scrambling [J]. Journal of Applied Sciences, 2019, 37(5): 631-642.
[12]	ZHAO Yan, ZHOU Xiaowei, SHEN Qi. Compact Image Hashing Algorithm Based on Opposite Color and Salient Region [J]. Journal of Applied Sciences, 2019, 37(5): 691-703.
[13]	ZHANG Yongsheng, TIAN Huawei, XIAO Yanhui, HAO Xinze, ZHANG Mingwang. PRNU Extraction Algorithm Based on Trilateral Weighted Sparse Coding Model [J]. Journal of Applied Sciences, 2019, 37(5): 704-710.
[14]	ZHU Yiming, CHEN Fan, HE Hongjie, CHEN Hongyou. Orthogonal GAN Information Hiding Model Based on Secret Information Driven [J]. Journal of Applied Sciences, 2019, 37(5): 721-732.
[15]	WANG Junxiang, MAO Ningxiong, ZHAO Yi, WANG Chuntao. Large Color Image Based High-Performance Reversible Data Hiding Hcheme for Various Capacities [J]. Journal of Applied Sciences, 2019, 37(5): 744-760.