融合BERT编码层的多粒度语义方面级情感分析模型

doi:10.3969/j.issn.0255-8297.2026.01.010

摘要/Abstract

摘要： 方面级情感分析旨在识别文本中针对特定方面的情感倾向，然而现有研究仍面临多重挑战：基于BERT的方面级情感分析研究存在语义过拟合、低层级语义利用不足的问题；自注意力机制存在局部信息丢失的问题；多编码层和多粒度语义的结构存在信息冗余问题。为此，提出一种融合BERT编码层的多粒度语义方面级情感分析模型（multi-granular semantic aspect-based sentiment analysis model with fusion of BERT encoding layers,MSBEL）。具体地，引入金字塔注意力机制，利用各个编码层的语义特征，并结合低层编码器以降低过拟合；通过多尺度门控卷积增强模型处理局部信息丢失的能力；使用余弦注意力突出与方面词相关的情感特征，从而减少信息冗余。t-SNE的可视化分析表明，MSBEL的情感表示聚类效果优于BERT。此外，在多个基准数据集上将本文模型与主流模型的性能进行了对比，结果显示：与LCF-BERT相比，本文模型在5个数据集上的F1分别提升了1.53%、3.94%、1.39%、6.68%、5.97%；与SenticGCN相比，本文模型的F1平均提升0.94%，最大提升2.12%；与ABSA-DeBERTa相比，本文模型的F1平均提升1.16%，最大提升4.20%，验证了本文模型在方面级情感分析任务上的有效性和优越性。

关键词: 方面级情感分析, 多粒度, BERT, 金字塔注意力机制, 多尺度门控卷积单元

Abstract: Aspect-based sentiment analysis (ABSA) aims to identify the sentiment polarity toward specific aspects within a text. However, existing research still faces multiple challenges: BERT-based approaches suffer from semantic overfitting and insufficient utilization of low-level semantic features; the self-attention mechanism is prone to losing local information; structures with multiple encoding layers and multi-granularity semantics lead to information redundancy. To address these issues, this paper proposed a multi-granularity semantic aspect-based sentiment analysis model with fusion of BERT encoding layers (MSBEL). The model introduced a pyramid attention mechanism to leverage semantic features from various encoding layers, and was combined with low-level encoders to mitigate overfitting. It employed multi-scale gated convolution to enhance its capability in handling local information loss and utilized cosine attention to highlight sentiment features relevant to aspect terms, thereby reducing information redundancy. t-SNE visualization demonstrates that the clustering effect of sentiment representations of MSBEL is superior to that of BERT. MSBEL was compared with mainstream models on multiple benchmark datasets. Compared with LCF-BERT, it achieves F1 improvements of 1.53%, 3.94%, 1.39%, 6.68%, and 5.97% on five datasets. In comparison with SenticGCN, it achieves an average increase of F1 by 0.94% and a maximum increase of 2.12%. Compared with ABSA-DeBERTa, MSBEL increases the F1 by 1.16% on average and achieves a maximum increase of 4.20%. These results validate the effectiveness and superiority of the proposed model for ABSA tasks.

Key words: aspect-based sentiment analysis, multi-granularity, BERT, pyramid attention mechanism, multi-scale gated convolutional unit

中图分类号:

TP391.1

徐凯, 池明得, 王崎, 李建州, 张辉. 融合BERT编码层的多粒度语义方面级情感分析模型[J]. 应用科学学报, 2026, 44(1): 149-165.

XU Kai, CHI Mingde, WANG Qi, LI Jianzhou, ZHANG Hui. Multi-granularity Semantic Aspect-Based Sentiment Analysis Model with Fusion of BERT Encoding Layers[J]. Journal of Applied Sciences, 2026, 44(1): 149-165.

参考文献

[1] 陈俊, 席宁丽, 李佳敏, 等. 融合Skip-Gram与R-SOPMI的教育领域情感词典构建[J]. 应用科学学报, 2023, 41(5): 870-880. Chen J, Xi N L, Li J M, et al. Constructing sentiment lexicon in the education field by integrating Skip-Gram and R-SOPMI [J]. Journal of Applied Sciences, 2023, 41(5): 870-880. (in Chinese).
[2] 艾均, 洪星琦. 融合标题情感和话题特征的新闻推荐算法[J]. 应用科学学报, 2024, 42(5): 810-822. Ai J, Hong X Q. News recommendation algorithm incorporating headline sentiment and topic characteristics [J]. Journal of Applied Sciences, 2024, 42(5): 810-822. (in Chinese).
[3] Tan K L, Lee C P, Lim K M. A survey of sentiment analysis: approaches, datasets, and future research [J]. Applied Sciences, 2023, 13(7): 4550.
[4] Hochreiter S, Schmidguber J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[5] Beck M, Pöppel K, Spanring M, et al. xLSTM: extended long short-term memory [DB/OL]. (2024-12-06) [2025-08-06]. https://arxiv.org/abs/2405.04517.
[6] Tang D Y, Qin B, Feng X C, et al. Effective LSTMs for target-dependent sentiment classification [DB/OL]. (2024-12-06) [2025-08-06]. https://arxiv.org/abs/1512.01100.
[7] Wang Y Q, Huang M L, Zhu X Y, et al. Attention-based LSTM for aspect-level sentiment classification [C]//Conference on Empirical Methods in Natural Language Processing, 2016: 606-615.
[8] 王亚新, 王亮, 王军. 基于上下文方面记忆网络的方面级情感分析[J]. 计算机工程与设计, 2024, 45(2): 443-451. Wang Y X, Wang L, Wang J. Aspect-based sentiment analysis based on contextual aspect memory network [J]. Computer Engineering and Design, 2024, 45(2): 443-451. (in Chinese).
[9] 韩虎, 吴渊航, 秦晓雅. 面向方面级情感分析的交互图注意力网络模型[J]. 电子与信息学报, 2021, 43(11): 3282-3290. Han H, Wu Y H, Qin X Y. An interactive graph attention networks model for aspect-level sentiment analysis [J]. Journal of Electronic & Information Technology, 2021, 43(11): 3282-3290. (in Chinese).
[10] Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]//Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[11] Brown T B, Mann B, Ryder N, et al. Language models are few-shot learners [C]//34th Conference on Neural Information Processing Systems (NeurIPS), 2020: 1877-1901.
[12] He P C, Gao J F, Chen W Z. DeBERTaV3: improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing [DB/OL]. (2023-03-24) [2025-08-06]. https://arxiv.org/abs/2111.09543.
[13] Bosley M, Jacobs-Harukawa M, Licht H, et al. Do we still need BERT in the age of GPT? Comparing the benefits of domain-adaptation and in-contextlearning approaches to using LLMs for political science research [EB/OL]. [2025-08-06]. https://mbosley.github.io/papers/bosley_harukawa_licht_hoyle_mpsa2023.pdf.
[14] Zhong Q H, Ding L, Liu J H, et al. Can ChatGPT understand too? A comparative study on ChatGPT and fine-tuned BERT [DB/OL]. (2023-03-24) [2025-08-06]. https://arxiv.org/abs/2302.10198.
[15] Boitel E, Mohasseb A, Haig E. A comparative analysis of GPT-3 and BERT models for textbased emotion recognition: performance, efficiency, and robustness [C]//22nd UK Workshop on Computational Intelligence (UKCI), 2024: 567-579.
[16] Touvron H, Lavril T, Izacard G, et al. LLaMA: open and efficient foundation language models [DB/OL]. (2023-02-27) [2025-08-06]. https://arxiv.org/abs/2302.13971.
[17] Song Y W, Wang J H, Jiang T, et al. Targeted sentiment classification with attentional encoder network [C]//28th International Conference on Artificial Neural Networks, 2019: 93- 103.
[18] Xu L L, Wang W M. Improving aspect-based sentiment analysis with contrastive learning [J]. Natural Language Processing Journal, 2023, 3: 100009.
[19] Zeng B Q, Yang H, Xu R Y, et al. LCF: a local context focus mechanism for aspect-based sentiment classification [J]. Applied Sciences, 2019, 9(16): 3389.
[20] Phan M H, Ogunbona P. Modelling context and syntactical features for aspect-based sentiment analysis [C]//58th Annual Meeting of the Association for Computational Linguistics, 2020: 3211-3220.
[21] 张心月, 刘蓉, 魏驰宇, 等. 融合提示知识的方面级情感分析方法[J]. 计算机应用, 2023, 43(9): 2753-2759. Zhang X Y, Liu R, Wei C Y, et al. Aspect-based sentiment analysis method with integrating prompt knowledge [J]. Journal of Computer Applications, 2023, 43(9): 2753-2759. (in Chinese)
[22] Li R F, Chen H, Feng F X, et al. Dual graph convolutional networks for aspect-based sentiment analysis [C]//59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 6319-6329.
[23] 韩虎, 孔博, 何勇禧, 等. 基于剪枝策略的知识增强方面级情感分析[J]. 华中科技大学学报(自然科学版), 2024, 52(11): 140-146. Han H, Kong B, He Y X, et al. Aspect based sentiment analysis based on knowledge enhancement of pruning strategy [J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2024, 52(11): 140-146. (in Chinese)
[24] Bajpai D J, Hanawal M K. CEEBERT: cross-domain inference in early exit BERT [DB/OL]. (2024-05-23) [2025-08-06]. https://arxiv.org/abs/2405.15039.
[25] Jawahar G, Sagot B, Seddah D. What does BERT learn about the structure of language? [C]//57th Annual Meeting of the Association for Computational Linguistics, 2019: 3651-3657.
[26] Song Y W, Wang J H, Liang Z W, et al. Utilizing BERT intermediate layers for aspect based sentiment analysis and natural language inference [DB/OL]. (2020-02-12) [2025-08-06]. https://arxiv.org/abs/2002.04815.
[27] Xiao Z G, Wu J R, Chen Q L, et al. BERT4GCN: using BERT intermediate layers to augment GCN for aspect-based sentiment classification [DB/OL]. (2021-10-01) [2025-08-06]. https://arxiv.org/abs/2110.00171.
[28] Feng A, Cai J Z, Gao Z J, et al. Aspect-level sentiment classification with fused local and global context [J]. Journal of Big Data, 2023, 10(1): 176.
[29] Dong L, Wei F R, Tan C Q, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification [C]//52nd Annual Meeting of the Association for Computational Linguistics, 2014: 49-54.
[30] Pontiki M, Galanis D, Pavlopoulos J, et al. SemEval-2014 task 4: aspect based sentiment analysis [C]//8th International Workshop on Semantic Evaluation (SemEval 2014), 2014: 27-35.
[31] Pontiki M, Galanis D, Papageorgiou H, et al. SemEval-2015 task 12: aspect based sentiment analysis [C]//9th International Workshop on Semantic Evaluation (SemEval 2015), 2015: 486-495.
[32] Pontiki M, Galanis D, Papageorgiou H, et al. SemEval-2016 task 5: aspect based sentiment analysis [C]//10th International Workshop on Semantic Evaluation (SemEval 2016), 2016: 19- 30.
[33] Ma D H, Li S J, Zhang X D, et al. Interactive attention networks for aspect-level sentiment classification [C]//26th International Joint Conference on Artificial Intelligence, 2017: 4068- 4074.
[34] Chen P, Sun Z, Bing L, et al. Recurrent attention network on memory for aspect sentiment analysis [C]//Conference on Empirical Methods in Natural Language Processing, 2017: 452-461.
[35] Liu Q, Zhang H B, Zeng Y F, et al. Content attention model for aspect based sentiment analysis [C]//27th World Wide Web Conference, 2018: 1023-1032.
[36] Huang B, Ou Y, Carley K M. Aspect level sentiment classification with attention-overattention neural networks [C]//11th International Conference on Social Computing, BehavioralCultural Modeling, and Prediction (SBP)/Conference of the Behavioral-Representation-inModeling-and-Simulation-Society (BRiMS), 2018: 197-206.
[37] Liang B, Su H, Gui L, et al. Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks [J]. Knowledge-Based Systems, 2022, 235: 107643.
[38] Marcacini R M, Silva E. Aspect-based sentiment analysis using BERT with disentangled attention [EB/OL]. [2025-08-06]. https://research.latinxinai.org/papers/icml/2021/ pdf/paper_28.pdf.
[39] Xu L L, Wang W M. Aspect-based sentiment classification with BERT and AI feedback [J]. Natural Language Processing Journal, 2025, 10: 100136.
[40] Aziz K, Ji D H, Chakrabarti P, et al. Unifying aspect-based sentiment analysis BERT and multi-layered graph convolutional networks for comprehensive sentiment dissection [J]. Scientific Reports, 2024, 14(1): 14646.