基于改进Graph2Seq的实体融合摘要生成模型

doi:10.3969/j.issn.1006-2475.2025.06.001

摘要/Abstract

摘要： 摘要：针对现有摘要生成模型占用计算资源大和对关键命名实体信息关注不足的问题，基于Graph2Seq模型提出一种融合实体和稀疏注意力的文摘生成模型（ESG2S）。首先，将原始文本构建为句法依存图，并进行实体节点增强，得到图数据；其次，将构建好的图数据送入编码器，进行文本结构的学习；最后，将编码后的图数据送入融合了对称散度增强稀疏注意力的LSTM解码器，生成多条摘要。在CNN/DM数据集上进行实验，结果表明本文模型效果优于近年的一些主流方法，并在实体信息保留上取得了成效，生成的摘要可读性和信息全面性更佳。

关键词: 关键词：关键词摘要生成, Graph2Seq, 命名实体, 稀疏注意力

Abstract: Abstract: In order to address the issues caused by high computational resource consumption and limited attention on key named entities， a novel summarization model named Entity-Sparse-Attention Graph-to-Sequence （ESG2S）， based on the Graph2Seq model， is proposed in this paper. Firstly， a graph data is created from a syntactic dependency graph enhanced by the extracted entity nodes from the original text. Secondly， this graph data is then input into an encoder to learn the textual structure. Finally， the encoded graph data is fed into an LSTM decoder integrated with Symmetric Divergence-Enhanced Sparse Attention to generate multiple summaries. Experiments on the CNN/DM dataset show that this model outperforms several recent mainstream methods and is effective in preserving entity information， resulting in summaries with better readability and comprehensiveness.

Key words:

Key words: keywords summarization, Graph2Seq, named entities, sparse attention

中图分类号:

TP391

陶源, 钱惠敏. 基于改进Graph2Seq的实体融合摘要生成模型[J]. 计算机与现代化, 2025, 0(06): 1-8.

TAO Yuan, QIAN Huimin. Entity-integrated Summarization Model Based on Improved Graph2Seq [J]. Computer and Modernization, 2025, 0(06): 1-8.

参考文献

［1］中国互联网络信息中心.第52次《中国互联网络发展状况统计报告》［EB/OL］. （2023-08-28）［2024-01-10］ https://www.cnnic.net.cn/n4/2023/0828/c88-10829.html.
［2］ SUTSKEVER I， VINYALS O， LE Q V. Sequence to sequence learning with neural networks ［C］// Proc of the 27th International Conference on Neural Information Processing Systems. MIT Press， 2014:3104-3112.
［3］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM， 2017:6000-6010.
［4］ XU K， WU L F， WANG Z G， et al. Graph2seq: Graph to sequence learning with attention-based neural network ［C］// Proceedings of International Conference on Learning Representations， ICLR， 2019:1-17.
［5］ LUHN H P. The automatic creation of literature abstracts［J］. IBM Journal of Research and Development， 1958，2（2）:159-165.
［6］ HENNIG L， UMBRATH W， WETZKER R. An ontology-based approach to text summarization ［C］// IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. IEEE， 2008，3:291-294.
［7］ RUSH A M， CHOPRA S， WESTON J. A neural attention model for sentence summarization ［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. ACL， 2015:379-389.
［8］ DEVLIN J， CHANG M W， LEE K， et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding［C］ // Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies. ACL， 2019:4171-4186.
［9］ ZHANG J Q， ZHAO Y， SALEH M， et al. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization［C］// International Conference on Machine Learning. PMLR， 2020:11328-11339.
［10］李伯涵，李红莲. 一种融合关键词的生成式摘要方法［J］. 计算机应用研究， 2021，38（11）:3289-3292.
［11］罗芳，汪竞航，何道森，等. 融合主题特征的文本自动摘要方法研究［J］. 计算机应用研究， 2021，38（1）:129-133.
［12］ ZAHEER M， GURUGANESH G， DUBEY K A， et al. Big bird: transformers for longer sequences ［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. NeurIPS， 2020:17283-17297.
［13］ BELTAGY I， PETERS M E， COHAN A. Longformer: The long-document transformer［J］. arXiv preprint arXiv:2004.05150， 2020.
［14］ KITAEV N， KAISER Ł， LEVSKAYA A. Reformer: The efficient transformer［J］. arXiv preprint arXiv:2001.04451， 2020.
［15］ CHEN Z D， QU Z， QUAN Y Y， et al. Dynamic N: M fine-grained structured sparse attention mechanism［C］// Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. ACM， 2023:369-379.
［16］ KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks ［C］// The 5th International Conference on Learning Representations. ICLR， 2017:1-14.
［17］ HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997，9（8）:1735-1780.
［18］ MANNING C D， SURDEANU M， BAUER J， et al. The Stanford CoreNLP natural language processing toolkit ［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. ACL， 2014:55-60.
［19］ ZHOU H Y， ZHANG S H， PENG J Q， et al. Informer: Beyond efficient transformer for long sequence time-series forecasting ［C］// Proceedings of the AAAI Conference on Artificial Intelligence. AAAI， 2021，35（12）:11106-11115.
［20］ LIN C Y. Rouge: A package for automatic evaluation of summaries ［C］// Text Summarization Branches Out. ACL， 2004:74-81.
［21］ LIU Y， LAPATA M. Text summarization with pretrained encoders［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. 2019:3730-3740.
［22］ ROTHE S， NARAYAN S， SEVERYN A. Leveraging pre-trained checkpoints for sequence generation tasks［J］. Transactions of the Association for Computational Linguistics， 2020，8:264-280.
［23］ SEE A， LIU P J， MANNING C D. Get to the point: Summarization with pointer-generator networks ［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1: Long Papers）. ACL， 2017:1073-1083.
［24］ GEHRMANN S， DENG Y， RUSH A M. Bottom-up abstractive summarization ［C］// Proceedings of Conference on Empirical Methods in Natural Language Processing. 2018:4098-4109.
［25］ SHARMA E， HUANG L Y， HU Z， et al. An entity-driven framework for abstractive summarization ［C］// Proceedings of Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. ACL， 2019:3280-3291.
［26］ HUANG L Y， WU L F， WANG L. Knowledge graph-augmented Abstractive summarization with semantic-driven cloze reward［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL， 2020:5094-5107.
［27］陈雪雯. 基于子词单元的深度学习摘要生成方法［J］. 计算机应用与软件， 2020，37（3）:202- 208.
［28］ ZHU C G， HINTHORN W C， XU R， et al. Enhancing factual consistency of abstractive summarization ［C］// Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL， 2021:718-733.
［29］杨涛，解庆，刘永坚，等. 主题感知的长文本自动摘要算法［J］. 计算机工程与应用， 2022，58（20）:165-173.
［30］ CHEN J Q. An entity-guided text summarization framework with relational heterogeneous graph neural network［J］. arXiv preprint arXiv:2302.03205， 2023.

[1]	王承先1, 赵晴2 . 基于CR-BACC模型的彝文命名实体识别方法[J]. 计算机与现代化, 2025, 0(05): 28-35.
[2]	赵盾1, 佘学兵2, 邬昌兴3. 基于BERT-BiLSTM-CRF党建领域命名实体识别[J]. 计算机与现代化, 2024, 0(09): 91-94.
[3]	王谭, 陈金广, 马丽丽. 融合词典信息和句子语义的中文命名实体识别[J]. 计算机与现代化, 2024, 0(03): 24-28.
[4]	乔璐, 孙有朝, 吴红兰. 面向飞机故障文本的信息抽取[J]. 计算机与现代化, 2024, 0(03): 61-66.
[5]	崔少国, 胡光平. 基于语义分割的嵌套命名实体识别方法[J]. 计算机与现代化, 2024, 0(02): 69-74.
[6]	郑立瑞, 肖晓霞, 邹北骥, 刘彬, 周展. 基于BERT的电子病历命名实体识别[J]. 计算机与现代化, 2024, 0(01): 87-91.
[7]	于韬, 张英, 拥措, . 基于小样本学习的藏文命名实体识别[J]. 计算机与现代化, 2023, 0(05): 13-19.
[8]	朱亚军, 拥措, 尼玛扎西, . 基于藏文BERT的藏医药医学实体识别[J]. 计算机与现代化, 2023, 0(01): 43-48.
[9]	于鹏, 陈钰枫, 徐金安, 张玉洁. 基于多任务学习的电子病历实体识别方法[J]. 计算机与现代化, 2022, 0(09): 40-50.
[10]	杨政, 蔡迪, 李慧斌. 基于层次化表示的电力文本命名实体识别和匹配算法[J]. 计算机与现代化, 2022, 0(05): 75-81.
[11]	李健, 张克亮, 唐亮, 夏榕璟, 任静静. 面向中文命名实体识别任务的数据增强[J]. 计算机与现代化, 2022, 0(04): 1-6.
[12]	王运乾, 王以松, 陈攀峰, 邹龙. 融合注意力机制的药用植物文本命名实体识别[J]. 计算机与现代化, 2021, 0(11): 100-105.
[13]	樊志强, 凌冬怡, 牛婵. 一种基于NLP的技术体制符合性审查方法[J]. 计算机与现代化, 2021, 0(09): 63-67.
[14]	李欣晔, 张承强, 周雄图, 郭太良, 张永爱. 多场景融合的细粒度图像描述生成算法[J]. 计算机与现代化, 2021, 0(09): 1-6.
[15]	张岑芳. 基于主动学习的命名实体识别算法[J]. 计算机与现代化, 2021, 0(07): 18-22.