计算机与现代化 ›› 2022, Vol. 0 ›› Issue (08): 1-6.

• 中文信息处理技术 •    下一篇

混合CTC/Attention模型在普通话识别中的应用

  

  1. (1.山东建筑大学信息与电气工程学院,山东济南250101;2.山东省智能建筑技术重点实验室,山东济南250101)
  • 出版日期:2022-08-22 发布日期:2022-08-22
  • 作者简介:许鸿奎(1966—),男,山东莱芜人,教授,博士,研究方向:模式识别与智能信息处理,E-mail: xhkui2009@163.com; 张子枫(1997—),男,山东泰安人,硕士研究生,研究方向:语音识别,E-mail: 1322848062@qq.com。
  • 基金资助:
    山东省重大科技创新工程项目(2019JZZY010120); 山东省重点研发计划项目(2019GSF111054)

Application of Hybrid CTC/Attention Model in Mandarin Recognition

  1. (1. School of Information and Electrical Engineering, Shandong Jianzhu University, Jinan 250101, China;
    2. Shandong Key Laboratory of Intelligent Buildings Technology, Jinan 250101, China)
  • Online:2022-08-22 Published:2022-08-22

摘要: 基于链接时序分类(Connectionist Temporal Classification, CTC)的端到端语音识别模型具有结构简单且能自动对齐的优点,但识别准确率有待进一步提高。本文引入注意力机制(Attention)构成混合CTC/Attention端到端模型,采用多任务学习方式,充分发挥CTC的对齐优势和Attention机制的上下文建模优势。实验结果表明,当选取80维FBank特征和3维pitch特征作为声学特征,选择VGG-双向长短时记忆网络(VGG-Bidirectional long short-time memory, VGG-BiLSTM)作为编码器应用于中文普通话识别时,该模型与基于CTC的端到端模型相比,字错误率下降约6.1%,外接语言模型后,字错误率进一步下降0.3%;与传统基线模型相比,字错误率也有大幅度下降。

关键词: 语音识别, 链接时序分类, 注意力机制, 端到端

Abstract: The end-to-end speech recognition model based on Connectionist Temporal Classification (CTC) has the advantages of simple structure and automatic alignment, but the recognition accuracy needs to be further improved. This paper introduces the attention mechanism to form a hybrid CTC/Attention end-to-end model. This method adopts the multi-task learning approach, combining the alignment advantage of CTC with the context modeling advantage of attention mechanism. The experimental results show that when the 80-dimensional FBank feature and the 3-dimensional pitch feature are selected as the acoustic features, and the VGG-Bidirectional long short-time memory network is selected as the encoder for Chinese Mandarin recognition, the character error rate of this hybrid model is reduced by about 6.1% compared with the end-to-end model based on CTC, after the external language model is connected, the character error rate is further reduced by 0.3%. Compared with the traditional baseline model, the character error rate also decreased significantly.

Key words: speech recognition, connectionist temporal classification, attention mechanism, end-to-end