基于问答交互的答案句选择算法

摘要/Abstract

摘要： 答案选择任务的精度对问答系统、文本处理等应用的效果具有重要的影响。针对答案选择模型问句与候选答案句语义信息和句子浅层特征利用不充分的问题，提出一种基于问答句交互的答案选择模型。给定问句Q和候选答句A，模型首先使用BiLSTM编码器对它们进行编码，然后针对问句Q使用Feed-Forward注意力机制得到句子编码；针对答句A，将问句Q和答句A的所有时间步输出两两进行匹配,根据匹配结果计算出答句的每个单词的权重，进而加权计算出答句的句子编码。最后，将问答句的句子编码经过聚合操作后输入全连接层，并与词共现特征相融合输出最终判断结果。在DBQA数据集上的实验结果表明，该模型与主流的Siamese结构的神经网络相比，能够有效地提升答案选择任务的效果。

关键词: 答案句选择, BiLSTM, Feed-Forward注意力机制, 问答句交互的注意力机制, 词共现

Abstract: The accuracy of answer selection task has an important influence on the application of question answering system, text processing and so on. In order to solve the problem that the semantic information and the shallow features of question and the candidate answer are not fully utilized in the answer selection model, an answer selection model based on the interaction of question and answer is proposed. Given question Q and candidate answer A, the model first uses BiLSTM encoder to encode them, concatenates the two directions encode results of each time step of Q and A, and then uses Feed-Forward attention to encode question sentence Q; all the time steps of question Q are matched with all the time steps of A. According to the matching results, the weight of each word of the answer sentence is calculated, and then the sentence code of the answer sentence is calculated according to the weight of each word of the answer sentence. Finally, the sentence coding of Q and A sentences is input into the full connection layer after the aggregation operation, and the final judgment result is output by fusion with the co-occurrence feature of words. The experimental results on the DBQA data set show that this model can effectively improve the effect of answer selection task compared with the mainstream Siamese neural network.

Key words: answer sentence selection, BiLSTM, Feed-Forward attention, attention mechanism of Q and A interaction, co-occurrence of words

侯佳正, 张绍阳, 鱼昆. 基于问答交互的答案句选择算法[J]. 计算机与现代化, 2021, 0(01): 120-126.

HOU Jia-zheng, ZHANG Shao-yang, YU Kun. Algorithm of Answer Sentence Selection Based on Q and A Interaction[J]. Computer and Modernization, 2021, 0(01): 120-126.

参考文献［26］

［1］	王东升,王卫民,王石,等. 面向限定领域问答系统的自然语言理解方法综述［J］. 计算机科学, 2017,44(8):1-8.
［2］	郭天翼,彭敏,伊穆兰,等. 自然语言处理领域中的自动问答研究进展［J］. 武汉大学学报(理学版), 2019,65(5):417-426.
［3］	YIH W T, CHANG M W, MEEK C, et al. Question answering using enhanced lexical semantic models［C］// Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013，1:1744-1753.
［4］	BARRN-CEDEO A, BONADIMAN D, DA SAN MARTINO G, et al. ConvKN at semeval-2016 task 3: Answer and question selection for question answering on Arabic and English fora［C］// Proceedings of the 10th International Workshop on Semantic Evaluation. 2016:896-903.
［5］	YAO X C, VAN DURME B. Information extraction over structured data: Question answering with freebase［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014,1:956-966.
［6］	BROMLEY J, GUYON I, LECUN Y, et al. Signature verification using a "Siamese" time delay neural network［C］// Proceedings of the 6th International Conference on Neural Information Processing Systems. 1993:737-744.
［7］	栾克鑫,孙承杰,刘秉权，等. 基于句内注意力机制的答案自动抽取方法［J］. 智能计算机与应用, 2017,7(5):87-91.
［8］	GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18(5-6):602-610.
［9］	〖KG-*5〗 TAN M, DOS SANTOS C, XIANG B, et al. LSTM-based deep learning models for non-factoid answer selection［C］// Proceedings of the International Conference on Learning Representations. 2016.
［10］	余本功,许庆堂. 基于协同注意力机制的答案选择算法研究［C］// 第十三届（2018）中国管理学年会论文集. 2018:534-540.〖HJ1.15mm〗
［11］	YIN W P, SCHTZE H, XIANG B, et al. ABCNN: Attention-based convolutional neural network for modeling sentence pairs［J］. Transactions of the Association for Computational Linguistics, 2016,4:259-272.
［12］	WANG S H, JIANG J. A compare-aggregate model for matching text sequences［J］. arXiv preprint arXiv:1611.01747, 2016.
［13］	WANG Z G, MI H T, ITTYCHERIAH A. Sentence similarity learning by lexical decomposition and composition［C］// Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2016:1340-1349.
［14］	HOCHREITER S, SCHMIDHUBER J. Long short-term memory［J］. Neural Computation, 1997,9(8):1735-1780.
［15］	CHEN Q, ZHU X D, LING Z H, et al. Enhanced LSTM for natural language inference［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2016:1657-1668.
［16］	WANG Z G, HAMZA W, FLORIAN R. Bilateral multi-perspective matching for natural language sentences［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017:4144-4150.
［17］	RAFFEL C, ELLIS D P W. Feed-forward networks with attention can solve some long-term memory problems［J］. arXiv preprint arXiv:1512.08756, 2015.
［18］	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013,2:3111-3119.
［19］	PENNINGTON J, SOCHER R, MANNING C D. Glove: Global vectors for word representation［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1532-1543.
［20］	CHO K, VAN MERRINBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1724-1734.
［21］	赵姗姗. 深度学习与多元特征相结合的答案选择排序研究［D］. 哈尔滨:哈尔滨工业大学, 2016.
［22］	张学武. 基于深度学习的候选答案句选择研究［D］. 广州:广东工业大学, 2019.
［23］	熊雪,刘秉权,吴翔虎. 基于注意力机制的答案选择方法研究［J］. 智能计算机与应用, 2018,8(6):90-93.
［24］	KINGMA D, BA J. Adam: A method for stochastic optimization［J］. arXiv preprint arXiv:1412.6980, 2014.
［25］	PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human language Technologies. 2018:2227-2237.
［26］	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding［J］. arXiv preprint arXiv:1810.04805, 2018.

[1]	魏鑫, 何小海, 滕奇志, 卿粼波, 陈洪刚. 基于BERT-BiLSTM-Attention混合模型的事件抽取方法[J]. 计算机与现代化, 2023, 0(04): 26-31.
[2]	王运乾, 王以松, 陈攀峰, 邹龙. 融合注意力机制的药用植物文本命名实体识别[J]. 计算机与现代化, 2021, 0(11): 100-105.
[3]	蒋万明1,2，郭春1,2，蒋朝惠1,2. 一种基于BiLSTM的低速率DDoS攻击检测方法[J]. 计算机与现代化, 2020, 0(05): 120-.
[4]	张苗苗，张玉洁，刘明童，徐金安，陈钰枫. 基于Gate机制与BiLSTMCRF的汉语语义角色标注[J]. 计算机与现代化, 2018, 0(04): 1-.
[5]	石爱萍. 一种基于语义距离的关键词获取方法 [J]. 计算机与现代化, 2010, 1(12): 18-20.