基于改进的MMR算法的新闻文本抽取式摘要方法

doi:10.3969/j.issn.0255-8297.2021.03.010

摘要/Abstract

摘要： 提出了基于最大边缘相关（maximal marginal relevance，MMR）的新闻摘要方法以及基于支持向量机（support vector machine，SVM）和MMR相结合的新闻摘要方法。其中，第1种方法是对传统MMR模型进行了改进，第2种方法使用了改进MMR模型对SVM分类结果进行了二次选择。实验表明：相比于传统MMR模型，该文提出的基于改进MMR的摘要方法和基于SVM-MMR的摘要方法的平均准确率分别提升了0.148、0.204，且基于MMR的新闻摘要方法的摘要效率约为基于SVM-MMR的摘要方法的3倍。改进的MMR算法更加适用于对摘要效率要求高的应用场景，特别是对长文本进行摘要。基于SVM-MMR的摘要方法则更适用于生成对文本内容覆盖相对全面的摘要。

关键词: 新闻摘要, 抽取式摘要, 冗余处理, 支持向量机, 最大边缘相关

Abstract: This paper proposes a news extraction method based on maximal marginal relevance (MMR) and a news extraction method based on support vector machine and maximal marginal relevance (SVM-MMR). The first method improves the traditional MMR news extraction method, and the second one uses the improved MMR news extraction method to make a second choice of the SVM classification results. Compared with the traditional MMR news extraction method, the average precision of MMR-based and SVMMMR-based news extraction methods are improved by 0.148 and 0.204, respectively. And the extraction efficiency of the MMR-based method is about 3 times of that of the SVMMMR method. The augmented MMR algorithm is more suitable for application scenarios that require high summarization efficiency, especially for long text summarization, while the SVM-MMR method is more suitable for generating a more comprehensive summary of the text content.

Key words: news extraction, extractive summarization, redundant processing, support vector machine (SVM), maximal marginal relevance (MMR)

中图分类号:

TP391

程琨, 李传艺, 贾欣欣, 葛季栋, 骆斌. 基于改进的MMR算法的新闻文本抽取式摘要方法[J]. 应用科学学报, 2021, 39(3): 443-442.

CHENG Kun, LI Chuanyi, JIA Xinxin, GE Jidong, LUO Bin. News Summarization Extracting Method Based on Improved MMR Algorithm[J]. Journal of Applied Sciences, 2021, 39(3): 443-442.

参考文献

[1] Mihalcea R, Tarau P. TextRank: bringing order into texts[C]//Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2004: 404-411.
[2] 荀静, 杨玉珍. 基于TextRank的文本情感摘要提取方法[J]. 计算机应用与软件, 2018, 35(10): 80-84. Gou J, Yang Y Z. Text emotion summarization extraction based on TextRank[J]. Computer Applications and Software, 2018, 35(10): 80-84. (in Chinese)
[3] 黄波, 刘传才. 基于加权TextRank的中文自动文本摘要[J]. 计算机应用研究, 2020, 37(2): 407-410. Huang B, Liu C C. Chinese automatic text summarization based on weighted TextRank[J]. Application Research of Computers, 2020, 37(2): 407-410. (in Chinese)
[4] Chakrabarti D, Punera K. Event summarization using tweets[C]//Proceedings of the 5th International AAAI Conference on Weblogs and Social Media, Barcelona, Catalonia, Spain, 2011: 66-73.
[5] Dohare S, Gupta V, Karnick H, et al. Unsupervised semantic abstractive summarization[C]//Proceedings of ACL 2018, Student Research Workshop, Melbourne, Australia, 2018: 74-83.
[6] Gillick D, Favre B. A scalable global model for summarization[C]//Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing, 2009: 10-18.
[7] Kgebck M, Mogren O, Tahmasebi N, et al. Extractive summarization using continuous vector space models[C]//Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC), Gothenburg, Sweden, 2014: 31-39.
[8] Lin C Y. Training a selection function for extraction[C]//Proceedings of the 8th International Conference on Information and Knowledge Management, Edinburgh, United Kingdom, 1999: 55-62.
[9] Conroy J M, O’leary D P. Text summarization via hidden Markov models[C]//Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New Orleans, Louisiana, USA, 2001: 406-407.
[10] Galley M. A skip-chain conditional random field for ranking meeting utterances by importance[C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA, United States, 2006: 364-372.
[11] Zhang J, Fung P. Speech summarization without lexical features for Mandarin broad-cast news[C]//Human Language Technologies 2007: the Conference of the North American Chapter of the Association for Computational Linguistics, Stroudsburg, PA, United States, 2007: 213-216.
[12] 张剑, 袁华强. Rhetorical-state SVM在抽取式语音摘要中的应用[J]. 科学技术与工程, 2013, 13(21): 6299-6301, 6305. Zhang J, Yuan H Q. Rhetorical-state SVM for extractive speech summarization[J]. Science Technology and Engineering, 2013, 13(21): 6299-6301, 6305. (in Chinese)
[13] 孟令阁, 马建芬, 张雪英. 基于主题的SVM与MMR融合的会议摘要技术[J]. 计算机工程与设计, 2016, 37(10): 2695-2699. Meng L G, Ma J F, Zhang X Y. Meeting summarization technology confused with SVM and MMR based on theme[J]. Computer Engineering and Design, 2016, 37(10): 2695-2699.
[14] Cao Z, Wei F, Li S, et al. Learning summary prior representation for extractive summarization[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 2015: 829-833.
[15] Nallapati R, Zhai F, Zhou B. SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI, 2017: 3075-3081.
[16] He R, Lee W S, Ng H T, et al. An unsupervised neural attention model for aspect extraction[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017: 388-397.
[17] 余珊珊, 苏锦钿, 李鹏飞. 基于改进的TextRank的自动摘要提取方法[J]. 计算机科学, 2016, 43(6): 240-247. Yu S S, Su J D, Li P F. Improved TextRank-based method for automatic summarization[J]. Computer Science, 2016, 43(6): 240-247. (in Chinese)
[18] 第一财经——汽车[EB/OL].[2020-10-26]. https://www.yicai.com/news/automobile/.
[19] 徐馨韬, 柴小丽, 谢彬, 等. 基于改进TextRank算法的中文文本摘要提取[J]. 计算机工程, 2019, 45(3): 273-277. Xu X T, Chai X L, Xie B, et al. Extraction of Chinese text summarization based on improved TextRank algorithm[J]. Computer Engineering, 2019, 45(3): 273-277. (in Chinese)