基于自注意力和胶囊网络的短文本情感分析

doi:10.3969/j.issn.1006-2475.2020.07.012

摘要/Abstract

摘要： 短文本的情感分析是一项具有挑战性的任务。针对传统的基于卷积神经网络和循环神经网络无法全面获取文本中蕴含的语义信息的缺点，本文提出一种使用多头自注意力层作为特征提取器，再以胶囊网络作为分类层的模型。该模型可以提取丰富的文本信息。在中文文本上进行实验结果表明，与传统深度学习方法相比，本文提出的模型提高了情感分析的精度，在小样本数据集和跨领域迁移中，相比传统方法精度都有较大的提高。

关键词: 情感分析, 自注意力机制, 胶囊网络, 小样本学习, 迁移学习

Abstract: Sentiment analysis of short texts is a challenging task. Aiming at the shortcomings of traditional convolutional neural networks and recurrent neural networks that can not fully obtain the semantic information contained in texts, this paper proposed a model that used the multi-head self-attention layer as the feature extractor and used the capsule network as the classification layer. The model can extract rich text information and has strong expressive ability. Experimental results on Chinese text showed that compared with the traditional deep learning method, the proposed model improved the accuracy of sentiment analysis. In the small dataset and cross-domain migration, compared with traditional method, the accuracy was greatly improved.

Key words: emotional analysis, self-attention mechanism, capsule network, small datasets learning, migration learning

中图分类号:

TP311

徐龙. 基于自注意力和胶囊网络的短文本情感分析[J]. 计算机与现代化, 2020, 0(07): 61-64.

XU Long. Short Text Sentiment Analysis Based on Self-attention and Capsule Network[J]. Computer and Modernization, 2020, 0(07): 61-64.

参考文献［20］

［1］	JIN H W, HUANG M L, ZHU X Y. Sentiment analysis with multi-source product reviews［C］// International Conference on Intelligent Computing. Springer Berlin Heidelberg, 2012:301-308.
［2］	陆文星,王燕飞. 中文文本情感分析研究综述［J］. 计算机应用研究, 2012,29(6):2014-2017.
［3］	PANG B, LEE L. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts［C］// Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. ACM, 2004:DOI:10.3115/1218955.1218990.
［4］	PANG B， LEE L, VAITHYANATHAN S. Thumbs up?Sentiment classification using machine learning techniques［C］// 〖JP2〗Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP). 2002:79-86.
［5］	MOHAMMAD S M, KIRITCHENKO S, ZHU X. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets［J］. arXiv preprint arXiv:1308.6242, 2013.
［6］	何炎祥,孙松涛,牛菲菲,等. 用于微博情感分析的一种情感语义增强的深度学习模型［J］. 计算机学报, 2017,40(4):773-790.
［7］	VO Q H, NGUYEN H T, LE B, et al. Multi-channel LSTM-CNN model for Vietnamese sentiment analysis［C］// 2017 the 9th International Conference on Knowledge and Systems Engineering (KSE). IEEE, 2017:DOI:10.1109/KSE.2017.8119429.
［8］	HAO Y Z, ZHENG Q H, LAN Y Y, et al. Improving Chinese sentiment analysis via segmentation-based representation using parallel CNN［C］// International Conference on Advanced Data Mining and Applications. Springer, 2017:668-680.
［9］	KIM Y. Convolutional neural networks for sentence classification［J］. arXiv preprint arXiv:1408.5882, 2014.
［10］	CAI G Y, XIA B B. Convolutional neural networks for multimedia sentiment analysis［C］// CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 2015:159-167.
［11］	SEVERYN A, MOSCHITTI A. Twitter sentiment analysis with deep convolutional neural networks［C］// Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2015:959-962.
［12］	BAGHERI H, ISLAM M J. Sentiment analysis of twitter data［J］. arXiv preprint arXiv:1711.10377, 2017.
［13］	LIN Z H, FENG M W, SANTOS C N, et al. A structured self-attentive sentence embedding［J］. arXiv preprint arXiv:1703.03130, 2017.
［14］	PAPPAS N, POPESCU-BELIS A. Multilingual hierarchical attention networks for document classification［J］. arXiv preprint arXiv:1707.00896, 2017.
［15］	RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving Language Understanding with Unsupervised Learning［R］. Technical Report, OpenAI, 2018.
［16］	DEVLIN J, CHANG M W, LEE K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding［J］. arXiv preprint arXiv:1810.04805, 2018.
［17］	SABOUR S, FROSST N, HINTON G E. Dynamic routing between capsules［C］// Advances in Neural Information Processing Systems. 2017:3856-3866.
［18］	HINTON G E, SABOUR S, FROSST N. Matrix capsules with EM routing［C］// The 6th International Conference on Learning Representations(ICLR). 2018:789.
［19］	张越,夏鸿斌. 基于弱监督预训练CNN模型的情感分析方法［J］. 计算机工程与应用, 2018,908(13):33-39.
［20］	MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space［J］. arXiv preprint arXiv:1301.3781, 2013.

[1]	李亚平, 王军防, 余红梅, 窦一民, 肖媛, 田继林. Regformer：基于稀疏注意力的输油管道水力压降预测方法[J]. 计算机与现代化, 2024, 0(01): 59-66.
[2]	胡崇佳, 刘金洲, 方立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79.
[3]	李诗月, 孟佳娜, 于玉海, 李雪莹, 许英傲. 基于知识增强的方面级情感分析方法[J]. 计算机与现代化, 2023, 0(10): 1-8.
[4]	张志霞, 谢宝强. 基于FCGA-LSTM与迁移学习的天然气负荷预测[J]. 计算机与现代化, 2023, 0(07): 7-12.
[5]	于韬, 张英, 拥措, . 基于小样本学习的藏文命名实体识别[J]. 计算机与现代化, 2023, 0(05): 13-19.
[6]	杨骏, 王劲林, 倪宏, 盛益强, . 工控网络异常检测中基于灵敏度的动态迁移算法[J]. 计算机与现代化, 2023, 0(05): 46-51.
[7]	陈晓雯, 石慧. 基于DWT-SVD与迁移学习的水印检测模型[J]. 计算机与现代化, 2023, 0(04): 111-117.
[8]	夏义春, 李汪根, 李豆豆, 葛英奎, 王志格. 结合注意力机制和图神经网络的CTR预估模型[J]. 计算机与现代化, 2023, 0(03): 29-37.
[9]	白旭光, 刘成忠, 韩俊英, 高嘉蒙, 陈俊康. 基于SE-ResNeXt的苹果叶片分类方法[J]. 计算机与现代化, 2023, 0(01): 18-23.
[10]	栗伟松, 汤敏芳, 何征岭, 王鹏, 杜利东, 方震, 陈贤祥. 基于自注意力机制和单导联心电信号的自动睡眠分期算法#br#[J]. 计算机与现代化, 2022, 0(12): 50-59.
[11]	何丽雯, 张锐驰. 基于深度学习的驾驶员分心行为识别[J]. 计算机与现代化, 2022, 0(06): 67-74.
[12]	徐亚军, 郭恩豪, 陈林, 司成可. 数据驱动的RF信号深度调制识别方法[J]. 计算机与现代化, 2022, 0(06): 80-86.
[13]	姚春华, 张学磊, 宋馨宇, 张举, 蔡佳志, 冯翱. 一种基于图卷积神经网络和依存分析的财经新闻情感分析方法[J]. 计算机与现代化, 2022, 0(05): 33-39.
[14]	冯敬翔. 基于迁移学习的卷积神经网络通道剪枝[J]. 计算机与现代化, 2021, 0(12): 13-18.
[15]	宋爽, 陆鑫达. 基于BERT与图像自注意力机制的文本匹配模型[J]. 计算机与现代化, 2021, 0(11): 12-16.