Existing graph neural network (GNN)-based question answering (QA) methods using pre-trained language models and knowledge graphs mainly focus on building knowledge graph subgraphs and reasoning processes. However, such methods ignore the semantic differences between question context and knowledge graphs, limiting their ability to deeply mine text representations. Moreover, they fail to comprehensively consider the varying contributions of these two representations to answer prediction. To address these challenges, this paper proposes an expert relearning reasoning QA method based on knowledge graphs and a gating mechanism. This method splices and fuses the question context representation with the inferred knowledge graph representation, and randomly assigns the fused representation vector to the expert network to relearn the entity semantic features associated with the question context and knowledge graph. By mining deeper hidden knowledge and incorporating the gating mechanism, the model accurately scores the question context and the inferred knowledge graph representation, dynamically adjusting their contribution to the answer prediction, and improving prediction accuracy. The proposed method was tested on the CommonsenseQA dataset and OpenBookQA dataset, achieving accuracy improvements of 2.08% and 1.23% over the QA-GNN method, respectively.
[1] Mollá D, Vicedo J L. Question answering in restricted domains: an overview [J]. Computational Linguistics, 2007, 33(1): 41-61.
[2] Bollacker K, Evans C, Paritosh P, et al. Freebase: a collaboratively created graph database for structuring human knowledge [C]//2008 ACM SIGMOD International Conference on Management of Data, 2008: 1247-1250.
[3] Speer R, Chin J, Havasi C. ConceptNet 5.5: an open multilingual graph of general knowledge [C]//31th AAAI Conference on Artificial Intelligence, 2017: 4444-4451.
[4] Scarselli F, Gori M, Tsoi A C, et al. The graph neural network model [J]. IEEE Transactions on Neural Networks, 2008, 20(1): 61-80.
[5] Lin B Y, Chen X, Chen J, et al. KagNet: knowledge-aware graph networks for commonsense reasoning [C]//2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019: 2829-2839.
[6] Feng Y, Chen X, Lin B Y, et al. Scalable multi-hop relational reasoning for knowledge-aware question answering [C]//2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020: 1295-1309.
[7] Yasunaga M, Ren H, Bosselut A, et al. QA-GNN: reasoning with language models and knowledge graphs for question answering [C]//2020 Conference on North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021: 535-546.
[8] Abujabal A, Yahya M, Riedewald M, et al. Automated template generation for question answering over knowledge graphs [C]//26th International Conference on World Wide Web, 2017: 1191-1200.
[9] Kapanipathi P, Abdelaziz I, Ravishankar S, et al. Question answering over knowledge bases by leveraging semantic parsing and neuro-symbolic reasoning [DB/OL]. 2020[2022-09-21]. http://arxiv.org/abs/2012.01707.
[10] Sun Y, Zhang L, Cheng G, et al. SPARQA: skeleton-based semantic parsing for complex questions over knowledge bases [C]//34th AAAI Conference on Artificial Intelligence, 2020: 8952-8959.
[11] Zhu S, Cheng X, Su S. Knowledge-based question answering by tree-to-sequence learning [J]. Neurocomputing, 2020, 372: 64-72.
[12] Maheshwari G, Trivedi P, Lukovnikov D, et al. Learning to rank query graphs for complex question answering over knowledge graphs [C]//18th International Semantic Web Conference. Springer, 2019: 487-504.
[13] Chen Y, Li H, Hua Y, et al. Formal query building with query structure prediction for complex question answering over knowledge base [C]//29th International Joint Conference on Artificial Intelligence, 2020: 3751-3758.
[14] Sun H, Dhingra B, Zaheer M, et al. Open domain question answering using early fusion of knowledge bases and text [C]//2018 Conference on Empirical Methods in Natural Language Processing, 2018: 4231-4242.
[15] Xiong W, Yu M, Chang S, et al. Improving question answering over incomplete KBs with knowledge-aware reader [C]//57th Meeting of the Association for Computational Linguistics, 2019: 4258-4264.
[16] Han J, Cheng B, Wang X. Open domain question answering based on text enhanced knowledge graph with hyperedge infusion [C]//Empirical Methods in Natural Language Processing, 2020: 1475-1481.
[17] Saxena A, Tripathi A, Talukdar P. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings [C]//58th Annual Meeting of the Association for Computational Linguistics, 2020: 4498-4507.
[18] Feng Y, Chen X, Lin B Y, et al. Scalable multi-hop relational reasoning for knowledge-aware question answering [C]//2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020: 1295-1309.
[19] Veličković P, Cucurull G, Casanova A, et al. Graph attention networks [C]//6th International Conference on Learning Representations (ICLR), 2018: 1-12.
[20] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]//32nd International Conference on Machine Learning(ICML), 2015: 448-456.
[21] Ma J, Zhao Z, Yi X, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts [C]//24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018: 1930-1939.
[22] Talmor A, Herzig J, Lourie N, et al. Commonsense QA: a question answering challenge targeting commonsense knowledge [C]//2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(NAACL-HLT), 2019: 4149-4158.
[23] Mihaylov T, Clark P, Khot T, et al. Can a suit of armor conduct electricity? a new dataset for open book question answering [C]//2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018: 2381-2391.
[24] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting [J]. The Journal of Machine Learning Research, 2014, 15(1): 1929- 1958.
[25] Liu L, Jiang H, He P, et al. On the variance of the adaptive learning rate and beyond [DB/OL].2021[2022-09-21]. http://arxiv.org/abs/1908.03265.
[26] Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks [C]//30th International Conference on Machine Learning (ICML), 2013: 1310-1318.
[27] Santoro A, Raposo D, Barrett D G, et al. A simple neural network module for relational reasoning [C]//30th Annual Conference on Neural Information Processing Systems, 2017: 4967- 4976.
[28] Schlichtkrull M, Kipf T N, Bloem P, et al. Modeling relational data with graph convolutional networks [C]//15th European Semantic Web Conference. Springer, 2018: 593-607.
[29] Wang X, Kapanipathi P, Musa R, et al. Improving natural language inference using external knowledge in the science questions domain [C]//AAAI Conference on Artificial Intelligence, 2019, 33(1): 7208-7215.
[30] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks [DB/OL]. 2017[2022-09-21]. http://arxiv.org/abs/1609.02907.
[31] Liu Y, Ott M, Goyal N, et al. Roberta: a robustly optimized bert pretraining approach [DB/OL]. 2019[2022-09-21]. http://arxiv.org/abs/1907.11692.
[32] Clark P, Etzioni O, Khashabi D, et al. From ‘F’ to ‘A’ on the N.Y. regents science exams: an overview of the aristo project [J]. AI Magazine, 2020, 41(4): 39-53.
[33] Chen Z, Parisa K. Dynamic relevance graph network for knowledge-aware question answering [C]//29th International Conference on Computational Linguistics (COLING), 2022: 1357-1366.
