CNN Text Classification Based on Topic Model Word Vectors

doi:10.3969/j.issn.1006-2475.2019.10.002

Abstract

Abstract: Mining information in Weibo text is of great significance to automatic question and answer, public opinion analysis and other applied research. The text classification study is the basis of text mining. This paper proposes to input simultaneously the text representations of Word2vec and LDA(Latent Dirichlet Allocation) into convolutional neural network model for high-level semantic feature abstraction and classification learning. The input word vectors can represent both the semantic information between the words and the theme of the text. First, We get the word vectors respectively based on the Word2vec model and LDA. Then the word vectors generated by the two models are cascaded to obtain their text matrix representations. Finally, We put the text matrices into the convolutional neural network simultaneously as two channels to classify the texts, and the effectiveness of the method is verified by experiments on Weibo data.

Key words: Word2Vec, LDA, text classification, convolutional neural network

CLC Number:

TP391

NIU Xue-ying. CNN Text Classification Based on Topic Model Word Vectors[J]. Computer and Modernization, 2019, 0(10): 7-.

References

［1］盛成成,朱勇,刘涛. 基于微博社交平台的舆情分析［J］. 智能计算机与应用, 2019,9(1):57-59.
［2］ ZHANG Y, JIN R, ZHOU Z H. Understanding bag-of-words model: A statistical framework［J］. International Journal of Machine Learning and Cybernetics, 2010,1(1-4):43-52.
［3］ SALTON G, WONG A, YANG C S. A vector space model for automatic indexing［J］. Communications of the ACM, 1975,18(11):613-620．
［4］ PHAN X H, NGUYEN L M, HORIGUCHI S．Learning to classify short and sparse text & Web with hidden topics from large-scale data collections［C］// Proceedings of the 17th International Conference on World Wide Web. 2008:91-100．
［5］张志飞,苗夺谦,高灿. 基于LDA主题模型的短文本分类方法［J］. 计算机应用, 2013,33(6):1587-1590.
［6］王细薇,樊兴华,赵军. 一种基于特征扩展的中文短文本分类方法［J］. 计算机应用, 2009,29(3):843-845．
［7］范云杰,刘怀亮. 基于维基百科的中文短文本分类研究［J］. 现代图书情报技术, 2012(3):47-52.
［8］周茜,赵明生,扈旻. 中文文本分类中的特征选择研究［J］. 中文信息学报, 2004(3):17-23.
［9］孟涛,王诚. 基于扩展短文本词特征向量的分类研究［J］. 计算机技术与发展, 2019,29(4):57-62.
［10］KRIZHEVSKY A, SUTSKEVER T, HINTON G. Imagenet classification with deep convolutional neural networks［C］// Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012:1097-1105.
［11］侯一民,周慧琼,王政一. 深度学习在语音识别中的研究进展综述［J］. 计算机应用研究, 2017,34(8):2241-2246.
［12］MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013:3111-3119.
［13］MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in Vector space ［J/OL］. Computer Science, 2013. ［2019-05-01］. http://export.arxiv.org/pdf/1301.3781.
［14］KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences［J/OL］. Computer Science, 2014.［2019-05-01］. http://de.arxiv.org/pdf/1404.2188.
［15］KIM Y. Convolutional neural networks for sentence classification［J/OL］. Computer Science, 2014.［2019-05-01］. https://arxiv.org/abs/1408.5882.
［16］韩栋,王春华,肖敏. 基于句子级学习改进CNN的短文本分类方法［J］. 计算机工程与设计, 2019,40(1):256-260.
［17］LAI S, XU L H, LIU K, et al. Recurrent convolutional neural networks for text classification［C］// Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015:2267-2273.
［18］ZHOU C, SUN C L, LIU Z Y, et al. A C-LSTM neural network for text classification［J/OL］. Computer Science, 2015.［2019-05-01］. http://arxiv.org/abs/1511.08630.
［19］YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification［C］// Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2016:1480-1489.
［20］张谦,高章敏,刘嘉勇. 基于Word2vec的微博短文本分类研究［J］. 信息网络安全, 2017(1):57-62.
［21］江大鹏. 基于词向量的短文本分类方法研究［D］. 杭州：浙江大学, 2015.
［22］汪静,罗浪,王德强. 基于Word2Vec的中文短文本分类问题研究［J］. 计算机系统应用, 2018,27(5):209-215.
［23］王盛,樊兴华,陈现麟．利用上下位关系的中文短文本分类［J］．计算机应用, 2010,30(3):603-606．

[1]	ZHANG Xiaodong1, BAI Guangzhi1, LI Min1, LI Haoyang2. Oil and Gas Well Production Prediction Model Based on Empirical Wavelet Transform [J]. Computer and Modernization, 2024, 0(12): 53-58.
[2]	LIU Baobao, YANG Jingjing, TAO Lu, WANG Heying . DSMSC Based on Attention Mechanism for Remote Sensing Image Scene Classification [J]. Computer and Modernization, 2024, 0(12): 72-77.
[3]	GAO Shuaipeng, WANG Yifan. Survey on Group-level Emotion Recognition in Images [J]. Computer and Modernization, 2024, 0(08): 98-107.
[4]	ZHOU Xianxi, MU Li. News Long Text Classification Model Based on Improved TF-IDF and AGLCNN [J]. Computer and Modernization, 2024, 0(08): 120-126.
[5]	LIU Cunli1, LEI Zhanzhan2, ZHENG Ao2. Circular Convolutional Neural Network-based Defect Detection Method for#br# Drainage Pipe Networks [J]. Computer and Modernization, 2024, 0(07): 26-35.
[6]	ZHANG Ke1, AI Zhongliang2, LIU Zhonglin3, GU Pingli1, LIU Xuelin4. Judicial Argumentation Understanding Method Based on Multiplet Loss [J]. Computer and Modernization, 2024, 0(06): 115-120.
[7]	LI Shan, WANG Linna, GAO Dingjia, XUAN Haibo. Multi-layer Bank-enterprise Converged Network Based on Graph Neural Network [J]. Computer and Modernization, 2024, 0(05): 27-32.
[8]	ZHONG Hailong1, 2, HE Yueshun1, HE Linlin1, CHEN JIE1, TIAN Ming3, ZHENG Ruiyin4. Cost-sensitive Convolutional Neural Network for Encrypted Traffic Classification#br# #br# [J]. Computer and Modernization, 2024, 0(05): 55-60.
[9]	GAO Geng1, XIAO Fengli2, YANG Fei1. Recognition of Hypopigmented Skin Diseases Based on Improved MobileNetV3-Small [J]. Computer and Modernization, 2024, 0(05): 120-126.
[10]	YOU Jiajing1, 2, HE Yueshun1, HE Linlin1, ZHONG Hailong1, 2. Encryption Traffic Classification Method Based on AHP-CNN [J]. Computer and Modernization, 2024, 0(04): 83-87.
[11]	XU Yue-wen1, LI Ming1, LI Li2. Image Classification of COVID-19 Based on Contrast Learning MocoV2 [J]. Computer and Modernization, 2024, 0(02): 81-87.
[12]	ZHOU Cheng-cheng, ZENG Qing-jun, YANG Kang, HU Jia-ming, HAN Chun-wei. EEG Recognition of Motor Imagination Based on Efficiency Channel Attention Module [J]. Computer and Modernization, 2023, 0(12): 19-23.
[13]	LIU Fu-qi, ZHANG Da, SONG Jian-hua, WANG Hai-dong. Fault Diagnosis of Hydraulic Systems Based on CNN-BiLSTM [J]. Computer and Modernization, 2023, 0(09): 10-19.
[14]	JIANG Lei, TANG Jian, YANG Chao-yue, LYU Ting-ting. Bearing Fault Diagnosis Based on CWGAN-GP and CNN [J]. Computer and Modernization, 2023, 0(07): 1-6.
[15]	XU Ye-tong, GENG Xin-zhe, ZHAO Wei-qiang, ZHANG Yue, NING Hai-long, LEI Tao. A Remote Sensing Image Change Detection Model Based on CNN-Transformer Hybrid Structure [J]. Computer and Modernization, 2023, 0(07): 79-85.