Question Classification Based on Hybrid Neural Network Model

doi:10.3969/j.issn.1006-2475.2018.09.001

Abstract

Abstract: The automatic question answering system gives fast and accurate answers to the questions proposed by the users in natural language, arousing widespread concern in academia and industry. By automatically determining the type of question, question classification task is of great significance to improve the accuracy of the question answering system. Based on the contextual information of the question and answer, combined with the respective advantages of convolutional neural networks and recurrent neural networks, this paper proposes a hybrid deep learning model. In addition, in order to strengthen the representation capacity of the question, this model adopts attention mechanism and enhances the generalization ability of the model. In this paper, we conduct a comparative experiment on 360 QA datasets, results show that this model has improved 1.6%~5.6% compared with the traditional method.

Key words: question classification, joint representation, deep learning, attention mechanism

CLC Number:

TP391

CHEN Ke-jin1,2,3, XU Guang-luan2,3, GUO Zhi2,3, LIANG Xiao2,3. Question Classification Based on Hybrid Neural Network Model[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2018.09.001.

References

［1］ Joachims T. Text categorization with support vector machines: Learning with many relevant features［C］// Proceedings of the 10th European Conference on Machine Learning. 1998:137-142.
［2］ Aikawa N, Sakai T, Yamana H. Community QA question classification: Is the asker looking for subjective answers or not?［J］. IPSJ Online Transactions, 2011,4:160-168.
［3］ Li Xin, Roth D. Learning question classifiers［C］// Proceedings of the 19th International Conference on Computational Linguistics. 2002,1:556-562.
［4］ Zhang D, Lee W S. Question classification using support vector machines［C］// Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2003:26-32.
［5］ Kim Y. Convolutional neural networks for sentence classification［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014:1746-1751.
［6］ Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE, 1998,86(11):2278-2324.
［7］ Ma Mingbo, Huang Liang, Xiang Bing, et al. Group sparse CNNs for question classification with answer sets［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017,2:335-340.
［8］ Gers F A, Schmidhuber J, Cummins F. Learning to forget: Continual prediction with LSTM［J］. Neural Computation, 2000,12(10):2451-2471.
［9］ Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate［DB/OL］. https://arxiv.org/pdf/1409.0473v7.pdf, 2016-05-19.
［10］Ding Zixiang, Xia Rui, Yu Jianfei, et al. Densely Connected Bidirectional LSTM with Applications to Sentence Classification［DB/OL］. https://arxiv.org/pdf/1802.00889v1.pdf, 2018-02-03.
［11］Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18(5-6):602-610.
［12］Yang Zichao, Yang Diyi, Dyer C, et al. Hierarchical attention networks for document classification［C］// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016:1480-1489.
［13］Jain L C, Medsker L R. Recurrent Neural Networks: Design and Applications［M］. CRC Press, 1999.
［14］Hochreiter S, Schmidhuber J. Long short-term memory［J］. Neural Computation, 1997,9(8):1735-1780.
［15］Graves A. Generating Sequences with Recurrent Neural Networks［DB/OL］. https://arxiv.org/pdf/1308.0850v5.pdf, 2014-06-05.
［16］张栋,李寿山,王晶晶. 基于问题与答案联合表示学习的半监督问题分类方法［J］. 中文信息学报, 2017,31(1):1-7.
［17］Mikolov T, Sutskever I, Chen Kai, et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 2013 Advances in Neural Information Processing Systems. 2013:3111-3119.
［18］Prechelt L. Early stopping-but when?［M］// Neural Networks: Tricks of the Trade. Springer, 1998:55-69.
［19］Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research, 2014,15(1):1929-1958.

[1]	HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation [J]. Computer and Modernization, 2024, 0(01): 74-79.
[2]	LIN Wei. Incremental News Recommendation Method Based on Self-supervised Learning and Data Replay [J]. Computer and Modernization, 2023, 0(12): 1-6.
[3]	LIANG Tian-kai, HUANG Kang-hua, LIU Kai-hang, LAN Lan, ZENG Bi. Deep Federated Image Classification Method Based on Bilateral Homomorphic Encryption [J]. Computer and Modernization, 2023, 0(12): 36-40.
[4]	QIU Kai-xing, FENG Guang. A Multi-label Image Classification Model Based on Dual Feature Attention [J]. Computer and Modernization, 2023, 0(12): 41-47.
[5]	ZHANG Bo-quan, MAI Hai-peng, CHEN Jia-min, Pang Jin-ju. White Matter Hyperintensities Segmentation Based on High Gray Value#br# Attention Mechanism [J]. Computer and Modernization, 2023, 0(12): 67-75.
[6]	LI Yan-man, WANG Bi-heng, ZHAO Ling-yan. Safety Helmet Detection Based on Lightweight YOLOv5 [J]. Computer and Modernization, 2023, 0(10): 59-64.
[7]	LI Shi-da, XIANG Jian-wen. A Weakened Joint Reinforcement Method to Improve Robustness of Image Recognition Models [J]. Computer and Modernization, 2023, 0(10): 70-76.
[8]	SHEN Jia-wei, LU Yi-ming, CHEN Xiao-yi, QIAN Mei-ling, LU Wei-zhong, . Review of Research on Human Behavior Detection Methods Based on Deep Learning [J]. Computer and Modernization, 2023, 0(09): 1-9.
[9]	LIU Chan-yi, HUANG Dan, XUE Lin-yan, WANG Tao, ZHU Tao, . COVID-19 X-ray Classification Based on Improved Efficientnet Network [J]. Computer and Modernization, 2023, 0(09): 94-99.
[10]	MA Guo-xiang, YANG Ling-fei, YAN Chuan-bo, ZHANG Zhi-hao, SUN Bing, WANG Xiao-rong. Ultrasonic Image Diagnosis of Hepatic Echinococcosis Based on Deep DenseNet Network [J]. Computer and Modernization, 2023, 0(09): 100-104.
[11]	NONG Hao-cheng, REN De-jun, REN Qiu-lin, LIU Peng-li, HUANG De-cheng. Surface Anomaly Detection Algorithm of Flexible Plastic Packaging Based on Improved ConvNeXt [J]. Computer and Modernization, 2023, 0(08): 12-17.
[12]	OUYANG Fei, WU Xu, XIANG Dong-sheng. Garbage Classification and Detection Method Based on Improved YOLOX [J]. Computer and Modernization, 2023, 0(08): 68-73.
[13]	HU Rui-jie, CHE Dou. Review of Infrared Small Target Detection [J]. Computer and Modernization, 2023, 0(08): 79-86.
[14]	JIANG Lei, TANG Jian, YANG Chao-yue, LYU Ting-ting. Bearing Fault Diagnosis Based on CWGAN-GP and CNN [J]. Computer and Modernization, 2023, 0(07): 1-6.
[15]	WANG Jia-chen, ZHANG Hong-xin, LIU Qing-hua, . CT Image Generation of Pneumonia Based on Generative Adversarial Network [J]. Computer and Modernization, 2023, 0(07): 20-24.