关键词: 文本摘要, 指针生成器网络, 对抗性扰动, 对比学习

Abstract: The model of abstractive text summarization based on the Seq2Seq framework has made great achievements. However, most of these models suffer from out-of-vocabulary， generated text repetition， and exposure bias. To tackle this problem, we propose a pointer generator network based on adversarial perturbation contrastive learning （PGN-CL） to model the text summarization generation process. As the basic structure， PGN is used for solving the problems of out-of-vocabulary and generated text repetition in this model as well as introducing Adversarial Perturbation Contrastive Learning as a new model training method to address exposure bias. In the model training process， we add perturbations to the target sequence and build a contrastive loss function to generate adversarial positive and negative samples. By this way， negative samples are similar to the target sequence in the embedding space but have large differences in semantic space， while the positive samples are similar to the target sequence in semantic space but have large differences in embedding space. These indistinguishable positive and negative samples can guide the model to learn the distinguishing features of these samples better in the feature space and obtain more accurate summary representation. The experiment result on the LCSTS dataset shows that the proposed model outperforms the comparative baselines on the ROUGE evaluation metric, demonstrating the effectiveness of the proposed model for summary quality improvement.

Key words: text summarization, pointer generator network, adversarial perturbation, contrastive learning