基于DTS-ILDA模型和关联过滤的新闻话题演化分析

doi:10.3969/j.issn.0255-8297.2017.05.009

Abstract

Abstract:

In topic evolution and tracking, as the size of time slices and the K value of the topic model are fxed, it is hard to locate important time turning points, which is prone to error topic correlation in the evolutionary analysis. To solve the problem, we propose an improved dynamic temporal segmentation-infnite latent Dirichlet allocation (DTS-ILDA) model and an associated fltering mechanism. The model combines an improved dynamic time segmentation algorithm with an infnite latent Dirichlet allocation (ILDA) model to extract topics. Dynamic time segmentation algorithm traverses the data set according to the time sequence, and then uses a contingency table to analysis the distribution of topics to measure the segmentation results and an ILDA model to extract K topics. In addition, an association fltering mechanism is proposed for error prone association in the evolutionary analysis. It removes weak association relationship. Finally, fve evolutionary relationships of right subtopic association are established according to the time sequence relationship. Experiments show that the presented method can effectively fnd important time points when the main content of the topic changes, preventing generation of meaningless topics. It can also reduce error-topic related interference, extracting exact deep relationship between the topics.

Key words: topic model, topic evolution, a fltering, temporal segment, infnite latent Dirichlet allocation (ILDA) mode

CLC Number:

TP391

GUO Xiao-li, ZHOU Zi-lan, LIU Yao-wei, DU Jian-hong, HUANG Yan. Analysis of News Topic Evolution Based on DTS-ILDA Model and Association Filtering[J]. Journal of Applied Sciences, 2017, 35(5): 634-646.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://www.jas.shu.edu.cn/EN/10.3969/j.issn.0255-8297.2017.05.009

https://www.jas.shu.edu.cn/EN/Y2017/V35/I5/634

[1] Cui W W, Liu S X, Tan L, Shi C L, Song Y Q, Gao Z J, Tong X, Qu H M. TextFlow:towards better understanding of evolving topics in text[J]. IEEE Transactions on Visualization & Computer Graphics, 2011, 17(12):2412-21.
[2] 曲朝阳,范旭东,曲楠,于华涛. 基于本体的智能电网文本知识获取模型[J]. 东北电力大学学报,2014, 34(5):60-68. Qu Z Y, Fan X D, Qu N, Yu H T. Smart grid text knowledge acquisition model based on ontology[J]. Journal of Northeast Dianli University, 2014, 34(5):60-68. (in Chinese)
[3] 曹丽娜,唐锡晋. 基于主题模型的BBS话题演化趋势分析[J]. 管理科学学报,2014, 17(11):109-121. Cao L N, Tang X J. Trends of BBS topics based on dynamic topic model[J]. Journal of Mangement Sciences in China, 2014, 17(11):109-121. (in Chinese)
[4] 洪宇,仓玉,朱巧明,姚建民,周国栋. 话题跟踪中静态和动态话题模型的核捕捉衰减[J]. 软件学报,2012, 23(5):1100-1119. Hong Y, Cang Y, Zhu Q M, Yao J M, Zhou G D. Descending kernel track of static and dynamic topic models in topic tracking[J]. Journal of Software, 2012, 23(5):1100-1119. (in Chinese)
[5] 徐戈,王厚峰. 自然语言处理中主题模型的发展[J]. 计算机学报,2011, 34(8):1423-1436. Xu G, Wang H F. The development of topic models in natural language processing[J]. Chinese Journal of Computers, 2011, 34(8):1423-1436. (in Chinese)
[6] 郭晓利,韩啸. 电网知识协同发现策略研究[J]. 东北电力大学学报,2014, 34(1):94-98. Guo X L, Han X. Grid knowledge collaborative discovery strategy research[J]. Journal of Northeast Dianli University, 2014, 34(1):94-98. (in Chinese)
[7] 杨玉珍,刘培玉,费绍栋,张成功. 融合扩展信息瓶颈理论的话题关联检测方法研究[J]. 自动化学报,2014, 40(3):471-479. Yang Y Z, Liu P Y, Fei S D, Zhang C G. A topic link detection method based on improved information bottleneck theory[J]. Acta Automatica Sinica, 2014, 40(3):471-479. (in Chinese)
[8] Hospedales T, Gong S, Xiang T. Video behavior mining using a dynamic topic model[J]. International Journal of Computer Vision, 2012, 98(3):303-323.
[9] Alsumait L, Barbar Daniel, Domeniconi C. On-line LDA:adaptive topic models for mining text streams with applications to topic detection and tracking[C]//IEEE International Conference on Data Mining. Pisa:IEEE, 2008:3-12.
[10] 胡艳丽,白亮,张维明. 一种话题演化建模与分析方法[J]. 自动化学报,2012, 38(10):1690-1697. Hu Y L, Bai L, Zhang W M. Modeling and analyzing topic evolution[J]. Acta Automatica Sinica, 2012, 38(10):1690-1697. (in Chinese)
[11] Wang X, Mccallum A. Topics over time:a non-Markov continuous-time model of topical trends[C]//ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Philadelphia:ACM, 2006:424-433.
[12] Hall D, Jurafsky D, Manning C D. Studying the history of ideas using topic models[C]//Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. PA:ACM, 2008:363-371.
[13] 赵旭剑,李波,杨春明,张晖,金培权,岳丽华,戴文锴. 一种基于特征演变的新闻话题演化挖掘方法[J]. 计算机学报,2014, 37(4):819-832. Zhao X J, Li B, Yang C M, Zhang H, Jing P Q, Yue L H, Dai W K. A topic evolution mining algorithm of news text based on feature evolving[J]. Chinese Journal of Computers, 2014, 37(4):819-832. (in Chinese)
[14] Brody S, Elhadad N. An unsupervised aspect-sentiment model for online reviews[C]//Human Language Technologies:Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, Los Angeles, California, USA. DBLP, 2013:804-812.
[15] Pan S M, Zhou M, Song Y Q, Qian W H, Wang F, Liu S X. Optimizing temporal topic segmentation for intelligent text visualization[C]//International Conference on Intelligent User Interfaces. Santa Monica:ACM, 2013:348-353.
[16] Gao Z, Song Y, Liu S, Wang H, Wei H, Chen Y, Cui W. Tracking and connecting topics via incremental hierarchical Dirichlet processes[C]//2011 IEEE 11th International Conference on Data Mining. Vancouver:IEEE, 2011:1056-1061.
[17] Gad S, Ramakrishnan N, Hampton K N, Kavanaugh A. Bridging the divide in democratic engagement:studying conversation patterns in advantaged and disadvantaged communities[C]//International Conference on Social Informatics. Alexandria:IEEE, 2012:165-176.
[18] 吕楠,罗军勇,刘尧,杨慧洁. 一种有效的事件演化分析算法[J]. 计算机应用研究,2009, 26(11):4101-4103. Lü N, Luo J Y, Liu Y, Yang H J. Effective event evolution analysis algorithm[J]. Application Research of Computers, 2009, 26(11):4101-4103. (in Chinese)
[19] 胡艳丽,白亮,张维明. 网络舆情中一种基于OLDA的在线话题演化方法[J]. 国防科技大学学报,2012, 34(1):150-154. Hu Y L, Bai L, Zhang W M. OLDA-based method for online topic evolution in network public opinion analysis[J]. Journal of National University of Defense Technology, 2012, 34(1):150-154. (in Chinese)
[20] Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation[J]. Journal of Machine Learning Research, 2003, 3:993-1022.
[21] Heinrich G. "Infnite LDA"-implementing the HDP with minimum code complexity[J]. Technical Note, 2011, 20(1):114-134.
[22] Teh Y W, Blei D M. Hierarchical Dirichlet processes[J]. Journal of the American Statistical Association, 2004, 101(467):1566-1581.
[23] Ding W, Chen C. Dynamic topic detection and tracking:a comparison of HDP, C-word, and cocitation methods[J]. Journal of the Association for Information Science & Technology, 2014, 65(10):2084-2097.
[24] 赵凡. 基于共词分析的学科主题动态跟踪相似算法改进研究[J]. 情报杂志,2010, 29(1):173-176. Zhao F. Research on similarity algorithm improvement of dynamic tracing disciplinary themes based on Co-word analysis[J]. Journal of Intelligence,2010, 29(1):173-176. (in Chinese)
[25] 李保利,杨星. 基于LDA模型和话题过滤的研究主题演化分析[J]. 小型微型计算机系,2012, 33(12):2738-2743. Li B L, Yang X. Analyzing research topic evolution with LDA and topic fltering[J]. Journal of Chinese Computer Systems, 2012, 33(12):2738-2743. (in Chinese)
[26] 曲朝阳,陈帅,杨帆,朱莉. 基于云计算技术的电力大数据预处理属性约简方法[J]. 电力系统自动化,2014, 38(8):67-71. Qu Z Y, Chen S, Yang F, Zhu L. An attribute reducing method for electric power big data preprocessing based on cloud computing technology[J]. Automation of Electric Power Systems, 2014, 38(8):67-71. (in Chinese)

Analysis of News Topic Evolution Based on DTS-ILDA Model and Association Filtering

PDF

Knowledge

Abstract

Cite this article

share this article

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	ZHAO Zixiao, WANG Hao, SHEN Tao, JIANG Shuting, ZHANG Siqi, LAI Hua, HUANG Yuxin, YU Zhengtao. Method for Extracting Chinese-Burmese Parallel Sentence Pairs Based on Language Feature Enhancement [J]. Journal of Applied Sciences, 2026, 44(3): 377-389.
[2]	LIAO Qing, CHEN Hongyou, LIU Chongyang, QU Lingfeng, DUAN Jiping, XIA Ping, TIAN Baodan, FAN Yong. Combustion Chamber Ignition Prediction Algorithm Based on AttResVGG Model [J]. Journal of Applied Sciences, 2026, 44(3): 390-408.
[3]	JIANG Zetao, JIN Xin, LENG Lu, ZHU Wencai. A Low-Light Few-Shot Object Detection Method Based on Feature Optimization [J]. Journal of Applied Sciences, 2026, 44(3): 422-436.
[4]	HE Xiaochen, DING Derui, LI Ming, WANG Fei, WANG Bo. Dual Attention-Incorporated Lightweight U-shaped Network for Lung Lesion Image Segmentation [J]. Journal of Applied Sciences, 2026, 44(3): 437-451.
[5]	LU Yongjie, AN Ping, HUANG Xinpeng, YANG Chao. Light Field Image Compression Based on Implicit Disparity Compensation [J]. Journal of Applied Sciences, 2026, 44(3): 452-464.
[6]	GUO Yan, WANG Haoran, HOU Songsong, DUAN Xuliang, MU Jiong. Cross-View Group Recommendation Algorithm Driven by Hypergraph Neural Networks [J]. Journal of Applied Sciences, 2026, 44(3): 486-502.
[7]	GAO Guangliang, LIANG Weichao, ZHU Tao, HONG Lei, XIA Lingling. Multi-modal Rumor Detection Method Fusing Image and Text Features [J]. Journal of Applied Sciences, 2026, 44(3): 503-514.
[8]	LI Yongzhen, MA Fuyuan, MA Shixuan, WANG Yuhan, WANG Ying. Network Community Detection Based on Structure-Enhanced Deep Clustering [J]. Journal of Applied Sciences, 2026, 44(1): 1-20.
[9]	LI Yinxiang, DU Wenyuan, XU Zhe, PENG Chen, YAN Jianqiang. Real-Time Travel Pattern Recognition Algorithm Based on Self Adaptive Pooling Enhanced Attention Mechanism [J]. Journal of Applied Sciences, 2026, 44(1): 21-33.
[10]	ZHANG Yuting, TENG Fei, YE Xiaoqing. Uncertain Knowledge Graph Reasoning Model Based on Gaussian Metric Learning [J]. Journal of Applied Sciences, 2026, 44(1): 50-66.
[11]	XIANG Ga, HU Yan, ZHANG Yangsen, SUN Lu, QI Rui, TAN Zicheng. Construction of Malware Knowledge Graph for Threat Intelligence Analysis [J]. Journal of Applied Sciences, 2026, 44(1): 67-82.
[12]	YI Huawei, SONG Shixi, WANG Yanfei, BAI Siyi. Federated Recommendation Algorithm Integrating Graph Neural Networks and Depth Graph Clustering [J]. Journal of Applied Sciences, 2026, 44(1): 83-96.
[13]	WU Wenqiang, CHEN Aibin, LI Xiaoyao. Forest Image Dehazing Based on Feature Fusion Attention and Contrastive Learning [J]. Journal of Applied Sciences, 2026, 44(1): 97-109.
[14]	ZHANG Xiaoming, FENG Zejia, WANG Huiyong, ZHANG Xiaojing. Explainable Learning Path Recommendation Based on Dynamic Attention Reinforcement Learning [J]. Journal of Applied Sciences, 2026, 44(1): 110-133.
[15]	JIN Zhengyang, YAN Shaohong, ZHANG Yanbo, YAO Xulong, TAO Zhigang, CHEN Zhiyuan. Three-Dimensional Fuzzy Clustering Algorithm Integrating Spatial Texture Features [J]. Journal of Applied Sciences, 2026, 44(1): 134-148.