计算机与现代化 ›› 2022, Vol. 0 ›› Issue (07): 121-126.

• 人工智能 • 上一篇    

加入奖励的GRU对抗网络文本生成模型

  

  1. (中国人民解放军海军工程大学电子工程学院指挥信息系统教研室,湖北武汉430030)
  • 出版日期:2022-07-25 发布日期:2022-07-25
  • 作者简介:彭鹏菲(1977—),男,湖北武汉人,副教授,博士,研究方向:指挥控制,E-mail: pengpengfei@126.com; 通信作者:周琳茹(1997—),女,河北沧州人,硕士研究生,研究方向:人工智能与模式识别,任务分析,E-mail: 1078591399@qq.com。

GRU Adversarial Network Text Generation Model with Reward

  1. (Department of Command Information System, College of Electronic Engineering, Naval University of Engineering, Wuhan 430030, China)
  • Online:2022-07-25 Published:2022-07-25

摘要: 针对目前生成对抗网络文本生成模型采用有监督形式造成的错误累计以及生成文本信息单一等问题,提出一种基于GRU生成对抗网络的文本生成模型,GRU生成器采用策略梯度进行参数更新,且该模型增加蒙特卡洛搜索推导生成样本序列。采用参数较少的GRU神经网络作为生成器和判别器,判别器的输出loss函数指导生成过程中的参数优化,以蒙特卡洛策略思想补充生成过程中的非完整序列,减少错误累计并增加文本生成信息的丰富性。引入门截断机制,用自定义函数替换GRU网络中的sigmoid函数,改进当前时刻的隐含变量的激活函数,改善原函数收敛速度较慢且容易产生梯度消失问题,使之更适应本文模型。仿真实验结果表明本文模型丰富了文本生成的多样性,提高了模型的收敛速度,验证了本模型的有效性。该模型有较好的应用性。

关键词: 生成对抗网络, 文本生成, GRU神经网络, 蒙特卡洛策略

Abstract: Aiming at the problems of accumulated errors caused by the supervised form of the current generative adversarial network text generation model  and the single generated text information, a text generation model based on GRU  generative adversarial network is proposed. The GRU generator uses rollout-policy to update parameters, and Monte Carlo search is added into the model to generate sample sequences. The GRU neural network with fewer parameters is used as the generator and the discriminator. The output loss function of the discriminator guides the parameter optimization in the generation process, and the Monte Carlo strategy is used to supplement the incomplete sequence in the generation process to reduce the accumulation of errors and increase the text richness of generated information. This paper introduces the gate truncation mechanism, replaces the sigmoid function in the GRU network with a custom function, improves the activation function of the implicit variable at the current time, and improves the slower convergence speed of the original function and the problem of gradient disappearance, making it more suitable for this model. The results of simulation experiments show that this model enriches the diversity of text generation, improves the convergence speed of the model, and proves the effectiveness of this model. The model has good applicability.

Key words: GAN, text generation, GRU(Gated Recurrent Unit) neural network, Monte Carlo strategy