基于深度学习和语法规约的需求文档命名实体识别

摘要/Abstract

摘要： 命名实体识别是自然语言处理中的一个关键。在需求文档中存在过长的实体：虚功能，使得普适的传统命名实体识别方法无法有效地识别得到完整的实体。本文针对需求文档实体识别模型进行深入研究，引入深度学习方法，提出基于深度残差网络（ResNet）的CNER方法与基于规则的方法相结合，进行针对中文需求文档的分词。本文的命名实体识别模型是一种编码-解码模型，使用带有注意力机制的双向长短期记忆网络（BiLSTM with attention）进行编码，得到分词后文本的上下文特征和句式特征，使用条件随机场（CRF）方法进行解码，再结合语法规约的干预进行需求文档实体识别。实验表明，所提方法在需求文档领域识别效果优于普适的传统方法。

关键词: 命名实体识别, CNER, 深度残差网络, 双向长短期记忆网络, 条件随机场, 语法规约

Abstract: Named entity recognition is particularly critical in natural language processing. There are overlong entities in the requirements document: virtual function, which makes it hard for pervasive traditional named entity recognition method to recognize entire entity. This paper conducts an in-depth research on the entity recognition model of requirements documents, introduces CNER method, which is based on Deep Residual Network (ResNet), to combine with the method based on grammatical regulations to perform word segmentation of Chinese requirements documents. This paper’s NER model is an encoder-decoder model, applies Bidirectional Long Short-Term Memory network (BiLSTM with attention) to encode, which obtains the context features and sentence pattern features of the text after word segmentation, employs conditional random field (CRF) method to decode, then identifies the requirements document entities with the intervention of grammatical regulations as a combination. Experiments show that the proposed method has better recognition effect than the pervasive traditional methods.

Key words: named entity recognition, CNER, ResNet, BiLSTM, CRF, grammatical regulations

许梦笛, 王金华. 基于深度学习和语法规约的需求文档命名实体识别[J]. 计算机与现代化, 2021, 0(01): 105-110.

XU Meng-di, WANG Jin-hua. Requirements Document Named Entity Recognition Based on Deep Learning and Grammatical Regulations [J]. Computer and Modernization, 2021, 0(01): 105-110.

参考文献［21］

［1］	LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition［C］// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016:260-270.
［2］	BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model［J］. Journal of Machine Learning Research, 2003，3:1137-1155.
［3］	MNIH A, HINTON G. A scalable hierarchical distributed language model［C］// Proceedings of the 21st International Conference on Neural Information Processing Systems. 2008:1081-1088.
［4］	CHEN W L, ZHANG Y J, ISAHARA H. Chinese named entity recognition with conditional random fields［C］// Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 2006:118-121.
［5］	HE J Z, WANG H F. Chinese named entity recognition and word segmentation based on character［C］// Proceedings of the 6th SIGHAN Workshop on Chinese Language Processing. 2008:128-132.
［6］	ZHANG Y, YANG J. Chinese NER using lattice LSTM［C］// Proceedings of the 56th Annual Meeting of the Assvciation for Computational Linguistics. 2018:1554-1564.
［7］	ZHU Y Y, WANG G X, KARLSSON B. CAN-NER: Convolutional attention network for Chinese named entity recognition［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:3384-3393.
［8］	DING R X, XIE P J, ZHANG X Y, et al. A neural multi-digraph model for Chinese NER with gazetteers［C］// Proceedings of the 57th Annual Meeting of the Assvciation for Computational Linguistics. 2019: 1462-1467.
［9］	LIU W, XU T G, XU Q H, et al. An encoding strategy based word-character LSTM for Chinese NER［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:2379-2389.
［10］	HAMMERTON J. Named entity recognition with long short-term memory［C］// Proceedings of the 7th Conference on Natural language Learning at HLT-NAACL. 2003,4:172-175.
［11］	COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch［J］. Journal of Machine Learning Research, 2011,12:2493-2537.
［12］	冯艳红,于红,孙庚,等. 基于词向量和条件随机场的领域术语识别方法［J］. 计算机应用, 2016,36(11):3146-3151．
［13］	冯艳红,于红,孙庚,等. 基于BLSTM的命名实体识别方法［J］. 计算机科学, 2018,45(2):261-268.
［14］	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［15］	SCHUSTER M, PALIWAL K K. Bidirectional recurrent neural networks［J］. IEEE Transactions on Signal Processing, 1997,45(11):2673-2681.
［16］	GRAVES A, SCHMIDHUBER J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures［J］. Neural Networks, 2005,18（5-6）: 602-610.
［17］	BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate［J］. arXiv:1409.0473, 2014.
［18］	MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space［J］. arXiv:1301.3781, 2013.
［19］	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013:3111-3119.
［20］	LING W, LUS T, MARUJO L, et al. Finding function in form: Compositional character models for open vocabulary word representation［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015:1520-1530.
［21］	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:4171-4186.

[1]	郑立瑞, 肖晓霞, 邹北骥, 刘彬, 周展. 基于BERT的电子病历命名实体识别[J]. 计算机与现代化, 2024, 0(01): 87-91.
[2]	于韬, 张英, 拥措, . 基于小样本学习的藏文命名实体识别[J]. 计算机与现代化, 2023, 0(05): 13-19.
[3]	朱亚军, 拥措, 尼玛扎西, . 基于藏文BERT的藏医药医学实体识别[J]. 计算机与现代化, 2023, 0(01): 43-48.
[4]	于鹏, 陈钰枫, 徐金安, 张玉洁. 基于多任务学习的电子病历实体识别方法[J]. 计算机与现代化, 2022, 0(09): 40-50.
[5]	周慧, 徐名海, 许晓东. 基于Attention-BIGRU-CRF的中文分词模型[J]. 计算机与现代化, 2022, 0(08): 7-12.
[6]	杨政, 蔡迪, 李慧斌. 基于层次化表示的电力文本命名实体识别和匹配算法[J]. 计算机与现代化, 2022, 0(05): 75-81.
[7]	李健, 张克亮, 唐亮, 夏榕璟, 任静静. 面向中文命名实体识别任务的数据增强[J]. 计算机与现代化, 2022, 0(04): 1-6.
[8]	王运乾, 王以松, 陈攀峰, 邹龙. 融合注意力机制的药用植物文本命名实体识别[J]. 计算机与现代化, 2021, 0(11): 100-105.
[9]	樊志强, 凌冬怡, 牛婵. 一种基于NLP的技术体制符合性审查方法[J]. 计算机与现代化, 2021, 0(09): 63-67.
[10]	李欣晔, 张承强, 周雄图, 郭太良, 张永爱. 多场景融合的细粒度图像描述生成算法[J]. 计算机与现代化, 2021, 0(09): 1-6.
[11]	张岑芳. 基于主动学习的命名实体识别算法[J]. 计算机与现代化, 2021, 0(07): 18-22.
[12]	朱岩, 张利, 王煜. 基于RoBERTa-WWM的中文电子病历命名实体识别[J]. 计算机与现代化, 2021, 0(02): 51-55.
[13]	李晓双, 韩立新, 李景仙, 周经纬. 基于优化残差网络的多模态音乐情感分类[J]. 计算机与现代化, 2020, 0(12): 83-89.
[14]	邵诗韵, 周宇, 杨蕾, 钟茂生, 戴芮, 赵家乐. 基于条件随机场的电力工程标书文本实体识别方法[J]. 计算机与现代化, 2020, 0(12): 72-77.
[15]	李凡, 白尚旺, 党伟超, 潘理虎. 基于Do-Bi-LSTM模型的电子政务文本相似度评估模型[J]. 计算机与现代化, 2020, 0(07): 71-75.