Chinese Word Segmentation Model Based on Attention-BIGRU-CRF

Abstract

Abstract: Natural language processing is an important branch of the development of artificial intelligence, and Chinese word segmentation is the first step of natural language processing. Improving the efficiency of Chinese word segmentation can improve the accuracy of the results of natural language processing. Therefore, an Attention-BIGRU-CRF-CRF model is proposed in this paper. Firstly, the Chinese text is transformed into vector form through word vector conversion, and then the BIGRU is used for serialization learning. Then, the attention mechanism is introduced to calculate the correlation between the input and output of BIGRU to obtain more accurate vector values, Finally, the vector value is spliced with the vector value serialized by BIGRU as the input of CRF layer, and the label prediction result is obtained. The simulation results show that the F1 values of Attention-BIGRU-CRF model in the corpus of people’s daily 2014 and MSRA are 97.34% and 98.25% respectively, and the word segmentation rate of processed text is 248.1 KB/s. Therefore, the model integrating attention mechanism and BIGRU-CRF network can not only improve the accuracy of word segmentation, but also improve the time efficiency of word segmentation.

Key words: BIGRU, CRF, attention mechanism, Chinese word segmentation

ZHOU Hui, XU Ming-hai, XU Xiao-dong. Chinese Word Segmentation Model Based on Attention-BIGRU-CRF[J]. Computer and Modernization, 2022, 0(08): 7-12.

References ［27］

［1］	KANTOR P B. Foundations of statistical natural language processing［J］. Information Retrieval, 2001,4（1）:80-81.
［2］	李霞婷. 基于改进型正反向最大匹配中文分词算法的研究［J］. 信息技术与信息化, 2015（6）: 211-212.
［3］	XUE N.Chinese word segmentation as character tagging［J］. Computational Linguistics Chinese Language Processing, 2003，8（1）:29-47.
［4］	丁振国,张卓,黎靖. 基于 Hash 结构的逆向最大匹配分词算法的改进［J］. 计算机工程与设计, 2008,29（12）:3208-3211.
［5］	陶伟. 警务应用中基于双向最大匹配法的中文分词算法实现［J］. 电子技术与软件工程, 2016（4）:153-155.
［6］	张玉茹. 中文分词算法之最大匹配算法的研究［J］. 现代计算机, 2011（16）:24-26.
［7］	王威. 基于统计学习的中文分词方法的研究［D］. 辽宁:东北大学, 2015.
［8］	杨贵军,徐雪,凤丽洲，等.基于最大匹配算法的似然导向中文分词方法［J］.统计与信息论坛, 2019,34（3）:18-23.
［9］	侯一民,周慧琼,王政一. 深度学习在语音识别中的研究进展综述［J］.计算机应用研究, 2017,34（8）:2241-2246.
［10］	HAN K, YU D, TASHEV I. Speech emotion recognition using deep neural network and extreme learning machine［J］. Interspeech, 2014:223-227.
［11］	卢宏涛,张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述［J］. 数据采集与处理, 2016,31（1）:1-17.
［12］	奚雪峰,周国栋. 面向自然语言处理的深度学习研究［J］. 自动化学报, 2016,42（10）:1445-1465.
［13］	BENGIO Y, DUCHARME R, VINCENT P, et al. Aneural probabilistic language model［J］. Journal of Machine Learning Research, 2003,3（6）:1137-1155.
［14］	COLLEBERT R, WESTON J,BOTTOU L, et al. Natural language processing（almost） from scratch［J］. Journal of Machine Learning Research, 2011,12（1）:2493-2537.
［15］	ZHENG X，CHEN H，XU T. Deep learning for Chinese word segmentation and POS tagging［C］// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle: Association for Computational Linguistics, 2013:647-657.
［16］	PEI W, GE T, CHANG B. Max-margin tensor neural network for Chinese word segmentation［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014:293-303.
［17］	BENGIO Y, SCHWENK H, SENECAL J S, et al. Neural probabilistic language models［M］. Innovations in Machine Learning. 2006:137-186.
［18］	CHEN X, IU X, ZHU C, et al. Gated recursive neural network for Chinese word segmentation［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015:567-572.
［19］	CHEN X, QIU X, ZHU C, et al. Long short-term memory neural networks for Chinese word segmentation［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015:1385-1394.
［20］	刁琦. 基于循环神经网络语言模型的中文分词研究［D］. 新疆:新疆农业大学,2017.
［21］	YAO Y, HUANG Z. Bi-directional LSTM recurrent neural network for Chinese word segmentation［C］// International Conference on Neural Information Processing. 2016:345-353.
［22］	JOZEFOWICZ R, ZAREMBA W, SUTSKEVER I.Anempirical exploration of recurrent network architectures［C］// Proceedings of the 32nd International Conference on International Conference on Machine Learning. 2015,37:2342-2350.
［23］	李雪莲,段鸿,许牧. 基于门循环单元神经网络的中文分词法［J］. 厦门大学学报（自然科学版）, 2017,56（2）:237-243.
［24］	车金立,唐力伟,邓士杰,等. 基于BI-GRU-CRF模型的中文分词法［J］.火力与指挥控制, 2019,44（9）:66-71.
［25］	黄丹丹,郭玉翠. 融合Attention机制的BI-LSTM-CRF中文分词模型［J］. 软件, 2018, 39（10）:260-266.
［26］	李文华. 中文分词算法在搜索引擎应用中的运用［J］.电脑知识与技术, 2021,17（6）:181-182.
［27］	刘佳萌. 语音交互智能音箱情感化设计策略研究［D］. 无锡:江南大学,2020.

[1]	WANG Qiu-yi, ZHOU Hao, ZHENG Ting-ting. Improved RetinaNet Target Detection Method for Power Equipment [J]. Computer and Modernization, 2024, 0(01): 47-52.
[2]	QIU Kai-xing, FENG Guang. A Multi-label Image Classification Model Based on Dual Feature Attention [J]. Computer and Modernization, 2023, 0(12): 41-47.
[3]	ZHANG Hao-yang, YIN Zi-ming, LE Jun-yi, SHEN Da-cong, SHU Yi-jun, YANG Zi-yi, . 3D-SPRNet: Segmentation Model of Gallbladder Cancer Based on Parallel Decoder and Double Attention Mechanism [J]. Computer and Modernization, 2023, 0(12): 59-66.
[4]	ZHANG Bo-quan, MAI Hai-peng, CHEN Jia-min, Pang Jin-ju. White Matter Hyperintensities Segmentation Based on High Gray Value#br# Attention Mechanism [J]. Computer and Modernization, 2023, 0(12): 67-75.
[5]	WANG Yu-hang, DONG Bao-liang, GONG Chao, SHANG Zhen-zhen, YAO Kang-ning. Dynamic Threat Assessment of Air Swarm Targets Based on Intent Recognition [J]. Computer and Modernization, 2023, 0(12): 100-104.
[6]	LUO Ming-jie, FENG Kai-ping. Lightweight Facial Expression Recognition Method Based on Sandglass Structure and Attention Mechanism [J]. Computer and Modernization, 2023, 0(11): 89-94.
[7]	ZHANG Jia-Qi, XU Qi-lei. Apple Defect Detection Algorithm Based on NAM-YOLO Network [J]. Computer and Modernization, 2023, 0(10): 53-58.
[8]	YE Si-jia, WEI Yan, DU Han-yu, DENG Jin-zhi. HRNet Image Semantic Segmentation Algorithm Combined with Attention Mechanism [J]. Computer and Modernization, 2023, 0(10): 65-69.
[9]	CHEN Jia-min, ZHANG Bo-quan, MAI Hai-peng. Hippocampus Segmentation Based on Feature Fusion [J]. Computer and Modernization, 2023, 0(08): 1-6.
[10]	LIU Xu, ZHA Ke-ke. An Environmental Target Recognition Method for Airport Special Vehicle Operation [J]. Computer and Modernization, 2023, 0(08): 18-24.
[11]	WANG Hong, GE Hong. Cross Modal Hash Retrieval Based on Attention Mechanism and Semantic Similarity [J]. Computer and Modernization, 2023, 0(08): 44-53.
[12]	OUYANG Fei, WU Xu, XIANG Dong-sheng. Garbage Classification and Detection Method Based on Improved YOLOX [J]. Computer and Modernization, 2023, 0(08): 68-73.
[13]	CUI Shao-guo, ZHANG Gang, WANG Ao-di. Deep Cross Network Recommendation Model Based on Attention Perception [J]. Computer and Modernization, 2023, 0(07): 54-60.
[14]	QIN Zhu-yuan, WU Hao-zhong, TAN Dai-qing, HAN Ai-qing, ZANG Hao, WANG Xuan, TANG Yan. Fine-grained Identification of Maidong Based on Multi-scale ResNet Combining Attention Mechanism [J]. Computer and Modernization, 2023, 0(07): 105-111.
[15]	GONG Xuan, GUO Zhong-hua, CHEN Wang. Remote Sensing Image Road Segmentation Based on CA-TransUNet [J]. Computer and Modernization, 2023, 0(07): 112-118.