Requirements Document Named Entity Recognition Based on Deep Learning and Grammatical Regulations

Abstract

Abstract: Named entity recognition is particularly critical in natural language processing. There are overlong entities in the requirements document: virtual function, which makes it hard for pervasive traditional named entity recognition method to recognize entire entity. This paper conducts an in-depth research on the entity recognition model of requirements documents, introduces CNER method, which is based on Deep Residual Network (ResNet), to combine with the method based on grammatical regulations to perform word segmentation of Chinese requirements documents. This paper’s NER model is an encoder-decoder model, applies Bidirectional Long Short-Term Memory network (BiLSTM with attention) to encode, which obtains the context features and sentence pattern features of the text after word segmentation, employs conditional random field (CRF) method to decode, then identifies the requirements document entities with the intervention of grammatical regulations as a combination. Experiments show that the proposed method has better recognition effect than the pervasive traditional methods.

Key words: named entity recognition, CNER, ResNet, BiLSTM, CRF, grammatical regulations

XU Meng-di, WANG Jin-hua. Requirements Document Named Entity Recognition Based on Deep Learning and Grammatical Regulations [J]. Computer and Modernization, 2021, 0(01): 105-110.

References

［1］ LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition［C］// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016:260-270.
［2］ BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model［J］. Journal of Machine Learning Research, 2003，3:1137-1155.
［3］ MNIH A, HINTON G. A scalable hierarchical distributed language model［C］// Proceedings of the 21st International Conference on Neural Information Processing Systems. 2008:1081-1088.
［4］ CHEN W L, ZHANG Y J, ISAHARA H. Chinese named entity recognition with conditional random fields［C］// Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 2006:118-121.
［5］ HE J Z, WANG H F. Chinese named entity recognition and word segmentation based on character［C］// Proceedings of the 6th SIGHAN Workshop on Chinese Language Processing. 2008:128-132.
［6］ ZHANG Y, YANG J. Chinese NER using lattice LSTM［C］// Proceedings of the 56th Annual Meeting of the Assvciation for Computational Linguistics. 2018:1554-1564.
［7］ ZHU Y Y, WANG G X, KARLSSON B. CAN-NER: Convolutional attention network for Chinese named entity recognition［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:3384-3393.
［8］ DING R X, XIE P J, ZHANG X Y, et al. A neural multi-digraph model for Chinese NER with gazetteers［C］// Proceedings of the 57th Annual Meeting of the Assvciation for Computational Linguistics. 2019: 1462-1467.
［9］ LIU W, XU T G, XU Q H, et al. An encoding strategy based word-character LSTM for Chinese NER［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:2379-2389.
［10］HAMMERTON J. Named entity recognition with long short-term memory［C］// Proceedings of the 7th Conference on Natural language Learning at HLT-NAACL. 2003,4:172-175.
［11］COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch［J］. Journal of Machine Learning Research, 2011,12:2493-2537.
［12］冯艳红,于红,孙庚,等. 基于词向量和条件随机场的领域术语识别方法［J］. 计算机应用, 2016,36(11):3146-3151．
［13］冯艳红,于红,孙庚,等. 基于BLSTM的命名实体识别方法［J］. 计算机科学, 2018,45(2):261-268.
［14］HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［15］SCHUSTER M, PALIWAL K K. Bidirectional recurrent neural networks［J］. IEEE Transactions on Signal Processing, 1997,45(11):2673-2681.
［16］GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18（5-6）: 602-610.
［17］BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate［J］. arXiv:1409.0473, 2014.
［18］MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space［J］. arXiv:1301.3781, 2013.
［19］MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013:3111-3119.
［20］LING W, LUS T, MARUJO L, et al. Finding function in form: Compositional character models for open vocabulary word representation［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015:1520-1530.
［21］DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:4171-4186.

[1]	GONG Yicheng1, 2, LIU Qing1, 2. Beijing Opera Binary Classification Based on RF-LCE-BiLSTM-Attention-AMSSA Model [J]. Computer and Modernization, 2024, 0(11): 7-12.
[2]	LI Junchao1, YOU Fei1, ZHANG Chao2, SU Lele2, GONG Yan2 . BiLSTM-Attention Prediction Model and Error Analysis #br# Based on Novel Multi-objective Coati Optimization Algorithm [J]. Computer and Modernization, 2024, 0(11): 70-76.
[3]	ZHAO Dun1, SHE Xuebing2, WU Changxing3. Named Entity Recognition in Field of Party Building Based on BERT-BiLSTM-CRF [J]. Computer and Modernization, 2024, 0(09): 91-94.
[4]	LAI Kun. Chinese Paper Invoice Text Recognition Method with Character Blurring [J]. Computer and Modernization, 2024, 0(08): 114-119.
[5]	FU Qi1, ZHANG Liyuan2, DAI Huan3. Features Analysis of Suicide Ideation Causes Based on Machine Learning [J]. Computer and Modernization, 2024, 0(04): 77-82.
[6]	QIAO Lu, SUN You-chao, WU Hong-lan. Information Extraction for Aircraft Fault Text [J]. Computer and Modernization, 2024, 0(03): 61-66.
[7]	CUI Shao-guo, HU Guang-ping. Nested Named Entity Recognition Based on Semantic Segmentation [J]. Computer and Modernization, 2024, 0(02): 69-74.
[8]	ZHENG Li-rui, XIAO Xiao-xia, ZOU Bei-ji, LIU Bin, ZHOU Zhan. Named Entity Recognition in Electronic Medical Record Based on BERT [J]. Computer and Modernization, 2024, 0(01): 87-91.
[9]	YU Tao, ZHANG Ying, Yong Tso, . Tibetan Named Entity Recognition Based on Small Sample Learning [J]. Computer and Modernization, 2023, 0(05): 13-19.
[10]	WEI Xin, HE Xiao-hai, TENG Qi-zhi, QING Lin-bo, CHEN Hong-gang. Event Extraction Method Based on BERT-BiLSTM-Attention Hybrid Model [J]. Computer and Modernization, 2023, 0(04): 26-31.
[11]	ZHU Ya-jun, Yong Tso, Nyima Tashi, . Tibetan Medical Entity Recognition Based on Tibetan BERT [J]. Computer and Modernization, 2023, 0(01): 43-48.
[12]	YU Peng, CHEN Yu-feng, XU Jin-an, ZHANG Yu-jie. Entity Recognition Method on EMR Based on Multi-task Learning [J]. Computer and Modernization, 2022, 0(09): 40-50.
[13]	ZHOU Hui, XU Ming-hai, XU Xiao-dong. Chinese Word Segmentation Model Based on Attention-BIGRU-CRF [J]. Computer and Modernization, 2022, 0(08): 7-12.
[14]	XU Ya-jun, GUO En-hao, CHEN Lin, SI Cheng-ke. A Data-driven Deep Modulation Identification Method for RF Signals [J]. Computer and Modernization, 2022, 0(06): 80-86.
[15]	FAN Hai-wei, QIN Jia-jie, SUN Huan, ZHANG Li-miao, LU Xin-siyu. Traffic Accident Text Information Extraction Model Based on BERT and BiGRU-CRF Fusion [J]. Computer and Modernization, 2022, 0(05): 10-15.

Requirements Document Named Entity Recognition Based on Deep Learning and Grammatical Regulations

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

Comments