融合BERT编码层的多粒度语义方面级情感分析模型

doi:10.3969/j.issn.0255-8297.2026.01.010

Abstract

Abstract: Aspect-based sentiment analysis (ABSA) aims to identify the sentiment polarity toward specific aspects within a text. However, existing research still faces multiple challenges: BERT-based approaches suffer from semantic overfitting and insufficient utilization of low-level semantic features; the self-attention mechanism is prone to losing local information; structures with multiple encoding layers and multi-granularity semantics lead to information redundancy. To address these issues, this paper proposed a multi-granularity semantic aspect-based sentiment analysis model with fusion of BERT encoding layers (MSBEL). The model introduced a pyramid attention mechanism to leverage semantic features from various encoding layers, and was combined with low-level encoders to mitigate overfitting. It employed multi-scale gated convolution to enhance its capability in handling local information loss and utilized cosine attention to highlight sentiment features relevant to aspect terms, thereby reducing information redundancy. t-SNE visualization demonstrates that the clustering effect of sentiment representations of MSBEL is superior to that of BERT. MSBEL was compared with mainstream models on multiple benchmark datasets. Compared with LCF-BERT, it achieves F1 improvements of 1.53%, 3.94%, 1.39%, 6.68%, and 5.97% on five datasets. In comparison with SenticGCN, it achieves an average increase of F1 by 0.94% and a maximum increase of 2.12%. Compared with ABSA-DeBERTa, MSBEL increases the F1 by 1.16% on average and achieves a maximum increase of 4.20%. These results validate the effectiveness and superiority of the proposed model for ABSA tasks.

Key words: aspect-based sentiment analysis, multi-granularity, BERT, pyramid attention mechanism, multi-scale gated convolutional unit

CLC Number:

TP391.1

XU Kai, CHI Mingde, WANG Qi, LI Jianzhou, ZHANG Hui. Multi-granularity Semantic Aspect-Based Sentiment Analysis Model with Fusion of BERT Encoding Layers[J]. Journal of Applied Sciences, 2026, 44(1): 149-165.

References

[1] 陈俊, 席宁丽, 李佳敏, 等. 融合Skip-Gram与R-SOPMI的教育领域情感词典构建[J]. 应用科学学报, 2023, 41(5): 870-880. Chen J, Xi N L, Li J M, et al. Constructing sentiment lexicon in the education field by integrating Skip-Gram and R-SOPMI [J]. Journal of Applied Sciences, 2023, 41(5): 870-880. (in Chinese).
[2] 艾均, 洪星琦. 融合标题情感和话题特征的新闻推荐算法[J]. 应用科学学报, 2024, 42(5): 810-822. Ai J, Hong X Q. News recommendation algorithm incorporating headline sentiment and topic characteristics [J]. Journal of Applied Sciences, 2024, 42(5): 810-822. (in Chinese).
[3] Tan K L, Lee C P, Lim K M. A survey of sentiment analysis: approaches, datasets, and future research [J]. Applied Sciences, 2023, 13(7): 4550.
[4] Hochreiter S, Schmidguber J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[5] Beck M, Pöppel K, Spanring M, et al. xLSTM: extended long short-term memory [DB/OL]. (2024-12-06) [2025-08-06]. https://arxiv.org/abs/2405.04517.
[6] Tang D Y, Qin B, Feng X C, et al. Effective LSTMs for target-dependent sentiment classification [DB/OL]. (2024-12-06) [2025-08-06]. https://arxiv.org/abs/1512.01100.
[7] Wang Y Q, Huang M L, Zhu X Y, et al. Attention-based LSTM for aspect-level sentiment classification [C]//Conference on Empirical Methods in Natural Language Processing, 2016: 606-615.
[8] 王亚新, 王亮, 王军. 基于上下文方面记忆网络的方面级情感分析[J]. 计算机工程与设计, 2024, 45(2): 443-451. Wang Y X, Wang L, Wang J. Aspect-based sentiment analysis based on contextual aspect memory network [J]. Computer Engineering and Design, 2024, 45(2): 443-451. (in Chinese).
[9] 韩虎, 吴渊航, 秦晓雅. 面向方面级情感分析的交互图注意力网络模型[J]. 电子与信息学报, 2021, 43(11): 3282-3290. Han H, Wu Y H, Qin X Y. An interactive graph attention networks model for aspect-level sentiment analysis [J]. Journal of Electronic & Information Technology, 2021, 43(11): 3282-3290. (in Chinese).
[10] Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]//Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[11] Brown T B, Mann B, Ryder N, et al. Language models are few-shot learners [C]//34th Conference on Neural Information Processing Systems (NeurIPS), 2020: 1877-1901.
[12] He P C, Gao J F, Chen W Z. DeBERTaV3: improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing [DB/OL]. (2023-03-24) [2025-08-06]. https://arxiv.org/abs/2111.09543.
[13] Bosley M, Jacobs-Harukawa M, Licht H, et al. Do we still need BERT in the age of GPT? Comparing the benefits of domain-adaptation and in-contextlearning approaches to using LLMs for political science research [EB/OL]. [2025-08-06]. https://mbosley.github.io/papers/bosley_harukawa_licht_hoyle_mpsa2023.pdf.
[14] Zhong Q H, Ding L, Liu J H, et al. Can ChatGPT understand too? A comparative study on ChatGPT and fine-tuned BERT [DB/OL]. (2023-03-24) [2025-08-06]. https://arxiv.org/abs/2302.10198.
[15] Boitel E, Mohasseb A, Haig E. A comparative analysis of GPT-3 and BERT models for textbased emotion recognition: performance, efficiency, and robustness [C]//22nd UK Workshop on Computational Intelligence (UKCI), 2024: 567-579.
[16] Touvron H, Lavril T, Izacard G, et al. LLaMA: open and efficient foundation language models [DB/OL]. (2023-02-27) [2025-08-06]. https://arxiv.org/abs/2302.13971.
[17] Song Y W, Wang J H, Jiang T, et al. Targeted sentiment classification with attentional encoder network [C]//28th International Conference on Artificial Neural Networks, 2019: 93- 103.
[18] Xu L L, Wang W M. Improving aspect-based sentiment analysis with contrastive learning [J]. Natural Language Processing Journal, 2023, 3: 100009.
[19] Zeng B Q, Yang H, Xu R Y, et al. LCF: a local context focus mechanism for aspect-based sentiment classification [J]. Applied Sciences, 2019, 9(16): 3389.
[20] Phan M H, Ogunbona P. Modelling context and syntactical features for aspect-based sentiment analysis [C]//58th Annual Meeting of the Association for Computational Linguistics, 2020: 3211-3220.
[21] 张心月, 刘蓉, 魏驰宇, 等. 融合提示知识的方面级情感分析方法[J]. 计算机应用, 2023, 43(9): 2753-2759. Zhang X Y, Liu R, Wei C Y, et al. Aspect-based sentiment analysis method with integrating prompt knowledge [J]. Journal of Computer Applications, 2023, 43(9): 2753-2759. (in Chinese)
[22] Li R F, Chen H, Feng F X, et al. Dual graph convolutional networks for aspect-based sentiment analysis [C]//59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 6319-6329.
[23] 韩虎, 孔博, 何勇禧, 等. 基于剪枝策略的知识增强方面级情感分析[J]. 华中科技大学学报(自然科学版), 2024, 52(11): 140-146. Han H, Kong B, He Y X, et al. Aspect based sentiment analysis based on knowledge enhancement of pruning strategy [J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2024, 52(11): 140-146. (in Chinese)
[24] Bajpai D J, Hanawal M K. CEEBERT: cross-domain inference in early exit BERT [DB/OL]. (2024-05-23) [2025-08-06]. https://arxiv.org/abs/2405.15039.
[25] Jawahar G, Sagot B, Seddah D. What does BERT learn about the structure of language? [C]//57th Annual Meeting of the Association for Computational Linguistics, 2019: 3651-3657.
[26] Song Y W, Wang J H, Liang Z W, et al. Utilizing BERT intermediate layers for aspect based sentiment analysis and natural language inference [DB/OL]. (2020-02-12) [2025-08-06]. https://arxiv.org/abs/2002.04815.
[27] Xiao Z G, Wu J R, Chen Q L, et al. BERT4GCN: using BERT intermediate layers to augment GCN for aspect-based sentiment classification [DB/OL]. (2021-10-01) [2025-08-06]. https://arxiv.org/abs/2110.00171.
[28] Feng A, Cai J Z, Gao Z J, et al. Aspect-level sentiment classification with fused local and global context [J]. Journal of Big Data, 2023, 10(1): 176.
[29] Dong L, Wei F R, Tan C Q, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification [C]//52nd Annual Meeting of the Association for Computational Linguistics, 2014: 49-54.
[30] Pontiki M, Galanis D, Pavlopoulos J, et al. SemEval-2014 task 4: aspect based sentiment analysis [C]//8th International Workshop on Semantic Evaluation (SemEval 2014), 2014: 27-35.
[31] Pontiki M, Galanis D, Papageorgiou H, et al. SemEval-2015 task 12: aspect based sentiment analysis [C]//9th International Workshop on Semantic Evaluation (SemEval 2015), 2015: 486-495.
[32] Pontiki M, Galanis D, Papageorgiou H, et al. SemEval-2016 task 5: aspect based sentiment analysis [C]//10th International Workshop on Semantic Evaluation (SemEval 2016), 2016: 19- 30.
[33] Ma D H, Li S J, Zhang X D, et al. Interactive attention networks for aspect-level sentiment classification [C]//26th International Joint Conference on Artificial Intelligence, 2017: 4068- 4074.
[34] Chen P, Sun Z, Bing L, et al. Recurrent attention network on memory for aspect sentiment analysis [C]//Conference on Empirical Methods in Natural Language Processing, 2017: 452-461.
[35] Liu Q, Zhang H B, Zeng Y F, et al. Content attention model for aspect based sentiment analysis [C]//27th World Wide Web Conference, 2018: 1023-1032.
[36] Huang B, Ou Y, Carley K M. Aspect level sentiment classification with attention-overattention neural networks [C]//11th International Conference on Social Computing, BehavioralCultural Modeling, and Prediction (SBP)/Conference of the Behavioral-Representation-inModeling-and-Simulation-Society (BRiMS), 2018: 197-206.
[37] Liang B, Su H, Gui L, et al. Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks [J]. Knowledge-Based Systems, 2022, 235: 107643.
[38] Marcacini R M, Silva E. Aspect-based sentiment analysis using BERT with disentangled attention [EB/OL]. [2025-08-06]. https://research.latinxinai.org/papers/icml/2021/ pdf/paper_28.pdf.
[39] Xu L L, Wang W M. Aspect-based sentiment classification with BERT and AI feedback [J]. Natural Language Processing Journal, 2025, 10: 100136.
[40] Aziz K, Ji D H, Chakrabarti P, et al. Unifying aspect-based sentiment analysis BERT and multi-layered graph convolutional networks for comprehensive sentiment dissection [J]. Scientific Reports, 2024, 14(1): 14646.

Multi-granularity Semantic Aspect-Based Sentiment Analysis Model with Fusion of BERT Encoding Layers

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 6

Recommended Articles

Metrics

Comments

[1]	LUO Hongmei, WANG Changjiang, YANG Peijie, GUAN Xiaoyan, ZHOU Xiaojie, YU Hang. Seismic Phase Clustering Analysis Technology Based on Multi-granularity Ensemble Learning [J]. Journal of Applied Sciences, 2025, 43(4): 643-655.
[2]	ZHAI Yayu, WANG Meihui, YANG Cheng, LU Jing, LI You. Three-Dimensional Measurement of Fringe-Gray Linear Projection Based on Hilbert Transform [J]. Journal of Applied Sciences, 2024, 42(2): 302-313.
[3]	ZHAO Yuhao, CHEN Yanping, HUANG Ruizhang, QING Yongbin. Chinese Event Trigger Extraction Based on Span Regression [J]. Journal of Applied Sciences, 2023, 41(1): 95-106.
[4]	LI Gang1, WAN You-chuan1, GUAN Yu-juan2. Multi-granularity Synthesis Segmentation of Remote Sensing Image Based on Quotient Space Theory [J]. Journal of Applied Sciences, 2011, 29(4): 390-396.
[5]	XU Xiang-li;ZHANG Li-biao;YU Zhe-zhou;ZHOU Chun-guang. Application of Multi-granularity Color Features in Image Retrieval [J]. Journal of Applied Sciences, 2009, 27(1): 56-61 .
[6]	YANG JINGAN. ERROR SENSITIVITY ANALYSIS AND CORRECTION FOR SURFACE NORMALS DETERMINED BY PHOTOMETRIC STEREO METHOD [J]. Journal of Applied Sciences, 1994, 12(3): 259-266.