Speech Emotion Recognition Based on ARCNN-GAP Network

Abstract

Abstract: Speech emotion recognition is the most direct way for computers to understand human emotions. It is an important channel to realize the intelligentization of human-computer interaction, but the performance of the recognition model needs to be further improved. To achieve this goal, a model ARCNN-GAP based on recurrent convolutional neural network is proposed for speech emotion recognition. Where the recurrent convolution layer has elastic path, which can ensure the depth of the network and the gradient return during optimization, and extract more effective features. The global average pooling is used to reduce the computational complexity and the risk of over-fitting. And the attention mechanism is employed to focus more on emotion-related features. The fusion features of prosodic features and spectral features are studied on CASIA and EMO-DB databases, and the recognition rates are 83.29% and 75.28% respectively. The experimental results show that the proposed model ARCNN-GAP features higher recognition performance and better generalization.

Key words: speech emotion recognition, recurrent convolutional neural network, global average pooling, attention mechanism

QIAN Jia-qi, HUANG He-ming, ZHANG Hui-yun, . Speech Emotion Recognition Based on ARCNN-GAP Network[J]. Computer and Modernization, 2021, 0(12): 91-95.

References

［1］李霞,卢官明,闫静杰,等. 多模态维度情感预测综述［J］. 自动化学报, 2018,44(12):2142-2159.
［2］李海峰,陈婧,马琳,等. 维度语音情感识别研究综述［J］. 软件学报, 2020,31(8):2465-2491.
［3］韩文静,李海峰,阮华斌,等. 语音情感识别研究进展综述［J］. 软件学报, 2014,25(1):37-50.
［4］赵腊生. 语音情感特征提取与识别方法研究［D］. 大连:大连理工大学, 2010.
［5］ BANSE R, SCHERER K R. Acoustic profiles in vocal emotion expression［J］. Journal of Personality and Social Psychology, 1996,70(3):614-636.
［6］ DELLAERT F, POLZIN T, WAIBEL A. Recognizing emotion in speech［C］// The 4th IEEE International Conference on Spoken Language Processing (ICSLP). 1996,3:1970-1973.
［7］ LUENGO I, NAVAS E, HERNAEZ I. Feature analysis and evaluation for automatic emotion identification in speech［J］. IEEE Transactions on Multimedia, 2010,12(6):490-501.
［8］ MIRSAMADI S, BARSOUM E, ZHANG C. Automatic speech emotion recognition using recurrent neural networks with local attention［C］// 2017 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). 2017:2227-2231.
［9］余莉萍,梁镇麟,梁瑞宇. 基于改进LSTM的儿童语音情感识别模型［J］. 计算机工程, 2020,46(6):40-49.
［10］SUN T W. End-to-end speech emotion recognition with gender information［J］. IEEE Access, 2020,8:152423-152438.
［11］柳长源,李文强,毕晓君. 基于RCNN-LSTM的脑电情感识别研究［J］. 自动化学报, 2020. DOI: 10.16383/j.aas.c190357.

［12］周飞燕,金林鹏,董军. 卷积神经网络研究综述［J］. 计算机学报, 2017,40(6):1229-1251.

［13］ZHANG T, ZHENG W M, CUI Z, et al. Spatial-temporal recurrent neural network for emotion recognition［J］. IEEE Transactions on Cybernetics, 2019,49(3):839-847.
［14］LIANG M, HU X L. Recurrent convolutional neural network for object recognition［C］// 2015 Conference on Computer Vision and Pattern Recognition (CVPR). 2015:3367-3375.
［15］梁鸣. 受神经科学启发的计算机识别和注意模型［D］. 北京:清华大学, 2016.
［16］赵明权. 基于图像场景和显著性信息的图像美感质量评估［D］. 广州:华南理工大学, 2017.
［17］赵泽宇,张卫强,刘加. 采用注意力机制和多任务训练的端到端无语音识别关键词检索［J］. 信号处理, 2020,36(6):839-851.
［18］ZHENG H Q, YANG Y R. An improved speech emotion recognition algorithm based on deep belief network［C］// 2019 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS). 2019:493-497.
［19］MUSTAQEEM, SAJJAD M, KWON S. Clustering-based speech emotion recognition by incorporating learned features and deep BiLSTM［J］. IEEE Access, 2020,8:79861-79875.
［20］张石清,赵知劲,雷必成,等. 结合音质特征和韵律特征的语音情感识别［J］. 电路与系统学报, 2009,14(4):120-123.
［21］LIU J X, LIU Z L, WANG L B, et al. Speech emotion recognition with local-global aware deep representation learning［C］// 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2020:7174-7178.
［22］王艳,胡维平. 基于BP特征选择的语音情感识别［J］. 微电子学与计算机, 2019,36(5):14-18.
［23］缪裕青,邹巍,刘同来,等. 基于参数迁移和卷积循环神经网络的语音情感识别［J］. 计算机工程与应用, 2019,55(10):135-140.
［24］陈嘉. 基于深度学习的语音情感识别［D］. 南京:南京邮电大学, 2019.

[1]	HE Sida, CHEN Pinghua. Intent-based Lightweight Self-Attention Network for Sequential Recommendation [J]. Computer and Modernization, 2024, 0(12): 1-9.
[2]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[3]	HUANG Tingpei1, MA Lubiao1, LI Shibao2, LIU Jianhang1. Gesture Recognition Method Based on WiFi and Prototypical Network [J]. Computer and Modernization, 2024, 0(12): 34-39.
[4]	ZHANG Xiaodong1, BAI Guangzhi1, LI Min1, LI Haoyang2. Oil and Gas Well Production Prediction Model Based on Empirical Wavelet Transform [J]. Computer and Modernization, 2024, 0(12): 53-58.
[5]	WANG Yanyuan, MAO Zhengchong. Detection and Recognition Algorithms for Chinese and English Scene Text Images [J]. Computer and Modernization, 2024, 0(12): 84-90.
[6]	LI Junchao1, YOU Fei1, ZHANG Chao2, SU Lele2, GONG Yan2 . BiLSTM-Attention Prediction Model and Error Analysis #br# Based on Novel Multi-objective Coati Optimization Algorithm [J]. Computer and Modernization, 2024, 0(11): 70-76.
[7]	ZHANG Yu1, 2, LI Jing1, 2, MA Ming1, 2, WANG Zhongxiang1, 2, SUN Yan1, 2. YOLOLW: A Novel Lightweight Object Detection Model [J]. Computer and Modernization, 2024, 0(11): 91-98.
[8]	QI Xian, LIU Daming, CHANG Jiaxin. Multi-view 3D Reconstruction Based on Improved Self-attention Mechanism [J]. Computer and Modernization, 2024, 0(11): 106-112.
[9]	YANG Jun1, HU Wei1, ZHU Wenfu2. Visual SLAM Loop Closure Detection Algorithm Based on Improved MobileNetV3 [J]. Computer and Modernization, 2024, 0(10): 21-26.
[10]	WEI Xuecheng1, JIANG Lingyun1, LI Yan2, HE Fei2. Improved Roadside Monocular View Small Target Detection Algorithm Based on YOLOv5 [J]. Computer and Modernization, 2024, 0(10): 27-34.
[11]	DU Mengjun1, LI Ang1, TONG Jun1, QIAN Jin1, KANG Kai1, WANG Ruoding1, JIN Wenxing2. Power Information Data Fusion Model Based on Improved Extreme Learning Algorithm [J]. Computer and Modernization, 2024, 0(10): 61-64.
[12]	YANG Shijun1, DI Guangyi1, GAO Jun1, CHEN Jianfei1, WANG Yaokun1, JI Xiaohan2. Sentiment Consistency Detection Based on Cross Modal Attention Fusion and#br# Information Perception [J]. Computer and Modernization, 2024, 0(10): 113-119.
[13]	HOU Congying, YANG Wengqing, WANG Zhao, CHENG Cong. Speech Enhancement Based on Time-frequency Self-attention Residual Temporal#br# Convolutional Networks [J]. Computer and Modernization, 2024, 0(09): 20-24.
[14]	ZHANG Ze1, ZHANG Jianquan2, 3, ZHOU Guopeng2, 3. Camera Module Defect Detection Based on Improved YOLOv8s [J]. Computer and Modernization, 2024, 0(09): 107-113.
[15]	ZHENG Shangpo1, CHEN Defu1, LI Jianli2, LIN Guoxian2, WANG Xingping3. Pedestrian Tracking Algorithm Based on Improved YOLOv5s and DeepSORT [J]. Computer and Modernization, 2024, 0(08): 54-58.