Entity-integrated Summarization Model Based on Improved Graph2Seq

doi:10.3969/j.issn.1006-2475.2025.06.001

Abstract

Abstract: Abstract: In order to address the issues caused by high computational resource consumption and limited attention on key named entities， a novel summarization model named Entity-Sparse-Attention Graph-to-Sequence （ESG2S）， based on the Graph2Seq model， is proposed in this paper. Firstly， a graph data is created from a syntactic dependency graph enhanced by the extracted entity nodes from the original text. Secondly， this graph data is then input into an encoder to learn the textual structure. Finally， the encoded graph data is fed into an LSTM decoder integrated with Symmetric Divergence-Enhanced Sparse Attention to generate multiple summaries. Experiments on the CNN/DM dataset show that this model outperforms several recent mainstream methods and is effective in preserving entity information， resulting in summaries with better readability and comprehensiveness.

Key words:

Key words: keywords summarization, Graph2Seq, named entities, sparse attention

CLC Number:

TP391

TAO Yuan, QIAN Huimin. Entity-integrated Summarization Model Based on Improved Graph2Seq [J]. Computer and Modernization, 2025, 0(06): 1-8.

References

［1］中国互联网络信息中心.第52次《中国互联网络发展状况统计报告》［EB/OL］. （2023-08-28）［2024-01-10］ https://www.cnnic.net.cn/n4/2023/0828/c88-10829.html.
［2］ SUTSKEVER I， VINYALS O， LE Q V. Sequence to sequence learning with neural networks ［C］// Proc of the 27th International Conference on Neural Information Processing Systems. MIT Press， 2014:3104-3112.
［3］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM， 2017:6000-6010.
［4］ XU K， WU L F， WANG Z G， et al. Graph2seq: Graph to sequence learning with attention-based neural network ［C］// Proceedings of International Conference on Learning Representations， ICLR， 2019:1-17.
［5］ LUHN H P. The automatic creation of literature abstracts［J］. IBM Journal of Research and Development， 1958，2（2）:159-165.
［6］ HENNIG L， UMBRATH W， WETZKER R. An ontology-based approach to text summarization ［C］// IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. IEEE， 2008，3:291-294.
［7］ RUSH A M， CHOPRA S， WESTON J. A neural attention model for sentence summarization ［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. ACL， 2015:379-389.
［8］ DEVLIN J， CHANG M W， LEE K， et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding［C］ // Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies. ACL， 2019:4171-4186.
［9］ ZHANG J Q， ZHAO Y， SALEH M， et al. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization［C］// International Conference on Machine Learning. PMLR， 2020:11328-11339.
［10］李伯涵，李红莲. 一种融合关键词的生成式摘要方法［J］. 计算机应用研究， 2021，38（11）:3289-3292.
［11］罗芳，汪竞航，何道森，等. 融合主题特征的文本自动摘要方法研究［J］. 计算机应用研究， 2021，38（1）:129-133.
［12］ ZAHEER M， GURUGANESH G， DUBEY K A， et al. Big bird: transformers for longer sequences ［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. NeurIPS， 2020:17283-17297.
［13］ BELTAGY I， PETERS M E， COHAN A. Longformer: The long-document transformer［J］. arXiv preprint arXiv:2004.05150， 2020.
［14］ KITAEV N， KAISER Ł， LEVSKAYA A. Reformer: The efficient transformer［J］. arXiv preprint arXiv:2001.04451， 2020.
［15］ CHEN Z D， QU Z， QUAN Y Y， et al. Dynamic N: M fine-grained structured sparse attention mechanism［C］// Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. ACM， 2023:369-379.
［16］ KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks ［C］// The 5th International Conference on Learning Representations. ICLR， 2017:1-14.
［17］ HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997，9（8）:1735-1780.
［18］ MANNING C D， SURDEANU M， BAUER J， et al. The Stanford CoreNLP natural language processing toolkit ［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. ACL， 2014:55-60.
［19］ ZHOU H Y， ZHANG S H， PENG J Q， et al. Informer: Beyond efficient transformer for long sequence time-series forecasting ［C］// Proceedings of the AAAI Conference on Artificial Intelligence. AAAI， 2021，35（12）:11106-11115.
［20］ LIN C Y. Rouge: A package for automatic evaluation of summaries ［C］// Text Summarization Branches Out. ACL， 2004:74-81.
［21］ LIU Y， LAPATA M. Text summarization with pretrained encoders［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. 2019:3730-3740.
［22］ ROTHE S， NARAYAN S， SEVERYN A. Leveraging pre-trained checkpoints for sequence generation tasks［J］. Transactions of the Association for Computational Linguistics， 2020，8:264-280.
［23］ SEE A， LIU P J， MANNING C D. Get to the point: Summarization with pointer-generator networks ［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1: Long Papers）. ACL， 2017:1073-1083.
［24］ GEHRMANN S， DENG Y， RUSH A M. Bottom-up abstractive summarization ［C］// Proceedings of Conference on Empirical Methods in Natural Language Processing. 2018:4098-4109.
［25］ SHARMA E， HUANG L Y， HU Z， et al. An entity-driven framework for abstractive summarization ［C］// Proceedings of Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. ACL， 2019:3280-3291.
［26］ HUANG L Y， WU L F， WANG L. Knowledge graph-augmented Abstractive summarization with semantic-driven cloze reward［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ACL， 2020:5094-5107.
［27］陈雪雯. 基于子词单元的深度学习摘要生成方法［J］. 计算机应用与软件， 2020，37（3）:202- 208.
［28］ ZHU C G， HINTHORN W C， XU R， et al. Enhancing factual consistency of abstractive summarization ［C］// Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ACL， 2021:718-733.
［29］杨涛，解庆，刘永坚，等. 主题感知的长文本自动摘要算法［J］. 计算机工程与应用， 2022，58（20）:165-173.
［30］ CHEN J Q. An entity-guided text summarization framework with relational heterogeneous graph neural network［J］. arXiv preprint arXiv:2302.03205， 2023.

[1]	HAO Rongrong1, MA Qiaomei1, TAN Yajun1, SHI Huanyin2. Complex Fire Detection Based on Deformable Convolutions and Attention Mechanisms [J]. Computer and Modernization, 2025, 0(06): 79-85.
[2]	GUO Xinyi1, JIANG Kai1, YANG Pengwei2, CHEN Geng2. Residential Microgrid Scheduling Algorithm Based on Deep Reinforcement Learning [J]. Computer and Modernization, 2025, 0(06): 106-113.
[3]	BAO Xuguang1, 2 , CHEN Zhiwei1, LI Qiaochen1, 2, JIANG Chengcheng1 . An Interactive Mining Approach for Spatial Co-location Patterns Incorporating#br# User Interest Preferences [J]. Computer and Modernization, 2025, 0(05): 10-20.
[4]	WANG Chengxian1, ZHAO Qing2. Yi Language Named Entity Recognition Method Based on CR-BACC Model [J]. Computer and Modernization, 2025, 0(05): 28-35.
[5]	CHI Biwei1, SUN Rui2. DDoS Attack Detection Method Based on Transformer Architecture [J]. Computer and Modernization, 2025, 0(05): 36-40.
[6]	MA Jian, WANG Yifei, MENG Li, HE Yunfei, YANG Fei. Label-independent Information Compression for Heterogeneous Graph Representation [J]. Computer and Modernization, 2025, 0(04): 36-41.
[7]	SUN Qining1, GUI Zhiming1, LIU Yanfang2, FAN Xinxin3, LU Yunfeng4. Graph Neural Network-based Multi-agent Reinforcement Learning for Adversarial Policy Detection Algorithm [J]. Computer and Modernization, 2025, 0(04): 42-49.
[8]	LIU Wenxin1, XU Wenhui1, CHEN Zhaoye1, GU Haiying1, WEN Cong2, YAO Yulong2, ZENG Xi2. Predictive Modeling of Ash Conveying in Thermal Power Plants Based on RFECV-XGBoost and SHAP [J]. Computer and Modernization, 2025, 0(04): 63-69.
[9]	WANG Jiale, SONG Wenai, FU Lizhen. Occupational Pneumoconiosis Screening Based on HA-Net Model [J]. Computer and Modernization, 2025, 0(04): 103-110.
[10]	LI Xiaomeng1, 2, JIANG Rong1, LIANG Zhihong3, YU Yimin1. Cross Border E-commerce Data Security Sharing Model Based on Zero Trust and Blockchain [J]. Computer and Modernization, 2025, 0(04): 111-118.
[11]	TANG Rui1, WU Jianchao1, CHEN Jianbo1, CHAI Jiang1, WANG Qian1, HE Yuchen2. Improved YOLOv8s Algorithm Based on GiraffeDet for Transmission Line Icing Detection [J]. Computer and Modernization, 2025, 0(03): 6-11.
[12]	CAO Lu, DING Cangfeng, MA Lerong, YAN Zhaoyao, YOU Hao. Multilevel Joint Graph Embedding for Lipophilic Molecular Classification [J]. Computer and Modernization, 2025, 0(03): 12-21.
[13]	ZHANG Taotao, XIE Jun, QIAO Pingjuan. Specific Emitter Identification Method Based on Multi-source Unsupervised Domain Adaptation [J]. Computer and Modernization, 2025, 0(03): 45-51.
[14]	SHAO Yeqin1, WANG Haiquan2, ZHOU Kunyang3, GUO Yudi2, SHI Quan1. One-stage Semi-supervised Object Detection by Reusing Unreliable Pseudo-labels [J]. Computer and Modernization, 2025, 0(03): 52-59.
[15]	BIAN Xinye1, XIE Dongmei1, WANG Ziling1, QU Zhijian1, YU Jiafeng2. sORF-BERT： A Method on Identifying Coding sORFs Based on Pre-trained Models [J]. Computer and Modernization, 2025, 0(03): 71-77.