融合注意力机制的药用植物文本命名实体识别

计算机与现代化 ›› 2021, Vol. 0 ›› Issue (11): 100-105.

融合注意力机制的药用植物文本命名实体识别

(1.贵州大学计算机科学与技术学院,贵州贵阳550025；2.公共大数据国家重点实验室（贵州大学）,贵州贵阳550025)

出版日期:2021-12-13 发布日期:2021-12-13
作者简介:王运乾(1991—)，男，贵州六盘水人，硕士研究生，研究方向：知识图谱,自然语言处理，E-mail: 1406436535@qq.com；王以松(1975—)，男，贵州铜仁人，教授，博士，研究方向：知识表示和推理,机器学习和人工智能；陈攀峰(1982—)，男，湖北黄冈人，博士研究生,研究方向：知识图谱,知识表示和推理；邹龙(1996—)，男，四川内江人，硕士研究生，研究方向：深度学习,知识表示。
基金资助:
国家自然科学基金资助项目（61976065, U1836205）

Named Entity Recognition of Medicinal Plant Texts Integrated with Attention Mechanism

(1. College of Computer Science and Technology, Guizhou University, Guiyang 550025, China；
2. State Key Laboratory of Public Big Data (Guizhou University), Guiyang 550025, China)

Online:2021-12-13 Published:2021-12-13

摘要/Abstract

摘要： 药用植物文本的命名实体识别对中医药领域的信息抽取和知识图谱构建起着重要作用。针对药用植物属性文本存在长序列语义稀疏的问题，提出一种基于注意力机制的双向长短时记忆网络(BiLSTM)和条件随机场(CRF)模型相结合的疾病实体识别方法（BiLSTM+ATT-CRF，BAC）。首先对药用植物属性文本进行预处理和半自动化标注构建数据集，并进行预训练得到低维词向量；然后将这些低维词向量输入BiLSTM网络中，得到双向语义依赖的特征向量；Attention层把注意力集中到与当前输出特征高度相关的信息上；最后通过条件随机场（CRF）算法获取最优的标签序列并解码输出。实验结果表明，BAC方法针对药用植物属性文本的长序列语义稀疏问题，疾病命名实体识别效果较传统方法更优。利用BAC方法训练好的模型从1680条文本句子中识别疾病命名实体，共抽取出1422个疾病实体。与药用植物名称进行匹配，共抽取出4316个药用植物治疗疾病的三元组数据。

关键词: 知识图谱, 注意力机制；双向长短时记忆网络(BiLSTM)；条件随机场(CRF)；疾病命名实体识别

Abstract: Named entity recognition of medicinal plant texts plays an important role in information extraction and knowledge graph construction in the field of traditional Chinese medicine. Aiming at the problem of long sequence semantic sparsity in medicinal plant attribute text, a disease entity recognition method BAC based on attention mechanism of BiLSTM and CRF model is proposed. Firstly, the medicinal plant attribute text is preprocessed and semi-automatic annotation is used to construct the medicinal plant knowledge data set, and the low-dimensional word vector is obtained by pre-training. Then， these vectors are fed into the attention-based BiLSTM network to obtain feature vectors that better represent disease entities. Finally, the optimal tag sequence is obtained by conditional random field (CRF) algorithm. The comparison of experimental results shows that the accuracy of BAC method reaches 93.78%, which is 4.46% higher than BiLSTM-CRF model, it can effectively improve the recognition effect of named entity of disease in medicinal plant attribute text. The model trained by BAC method is used to identify disease named entities from 1680 text sentences, and a total of 1422 disease entities are extracted. By matching with the names of medicinal plants, a total of 4316 triples of the relationship between medicinal plants name and diseases entities are extracted.

Key words: knowledge graph, attention mechanism, bidirectional long-short term memory network (BiLSTM), conditional random field (CRF), disease named entity recognition

王运乾, 王以松, 陈攀峰, 邹龙. 融合注意力机制的药用植物文本命名实体识别[J]. 计算机与现代化, 2021, 0(11): 100-105.

WANG Yun-qian, WANG Yi-song, CHEN Pan-feng, ZOU Long. Named Entity Recognition of Medicinal Plant Texts Integrated with Attention Mechanism[J]. Computer and Modernization, 2021, 0(11): 100-105.

参考文献

［1］ BERNERS-LEE T, HENDLER J, LASSILA O. The semantic Web［J］. Scientific American, 2001,284(5):34-43.
［2］ SINGHAL A. Introducing the knowledge graph: Things, not strings［DB/OL］.(2012-05-16)［2020-12-03］. https://www.blog.google/products/search/introducing-knowledge-graph-things-not/.
［3］ CHINCHOR N. MUC-6 named entity task definition［C］// Proceedings of the 6th Message Understanding Conference. 1995:317-332.
［4］孙镇，王惠临. 命名实体识别研究进展综述［J］. 现代图书情报技术, 2010(6):42-47.
［5］ ZHOU G D, SU J. Named entity recognition using an HMM-based Chunk Tagger［C］// Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. 2002:473-480.
［6］ ISOZAKI H, KAZAWA H. Efficient support vector classifiers for named entity recognition［C］// Proceedings of the 19th International Conference on Computational Linguistics. 2002. DOI: 10.3115/1072228.1072282.
［7］ MCCALLUM A, LI W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons［C］// Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL. 2003:188-191.
［8］ DEVLIN J， CHANG M W， LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(HLT-NAACL). 2019:4171-4186.
［9］ YANG Z L， DAI Z H， YANG Y M， et al. XLNet:Generalized autoregressive pretraining for language understanding［C］// Advances in Neural Information Processing Systems 32(NeurIPS 2019). 2019:5754-5764.
［10］冯蕴天,张宏军,郝文宁. 面向军事文本的命名实体识别［J］. 计算机科学, 2015,42(7):15-18.
［11］李想,魏小红,贾璐,等. 基于条件随机场的农作物病虫害及农药命名实体识别［J］. 农业机械学报, 2017,48(S1):178-185.
［12］周详,李少波,杨观赐. 服装类商品属性实体识别［J］. 计算机应用, 2015,35(7):1945-1949.
［13］JI S X, PAN S R, CAMBRIA E, et al. A survey on knowledge graph: Representation, acquisition and applications［J］. arXiv preprint arXiv:2002.00388, 2020.
［14］CHIU J P, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs［J］. arXiv preprint arXiv: 1511.08308, 2015.
［15］XIA C Y, ZHANG C W, YANG T, et al. Multi-grained named entity recognition［J］. arXiv preprint arXiv: 1906.08449, 2019.
［16］XU G H, WANG C Y, HE X F. Improving clinical named entity recognition with global neural attention［C］// Proceedings of the 2nd International Joint Conference on Web and Big Data. 2018:264-279.
［17］HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging［J］. arXiv preprint arXiv: 1508.01991, 2015.
［18］ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016:207-212.
［19］杨红梅，李琳，杨日东，等. 基于双向LSTM神经网络电子病历命名实体的识别模型［J］. 中国组织工程研究， 2018，22（20）：3237-3242.
［20］栗冬冬. 面向医疗领域的命名实体识别［D］. 哈尔滨：哈尔滨理工大学， 2018.
［21］李冬梅,檀稳. 植物属性文本的命名实体识别方法研究［J］. 计算机科学与探索, 2019,13(12):2085-2093.
［22］ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks ［J］. arXiv preprint arXiv:1805.08318, 2018.
［23］ZENG Y, YANG H H, FENG Y S, et al. A convolution BiLSTM neural network model for Chinese event extraction［C］// Proceedings of the 2016 International Conference on Computer Processing of Oriental Languages. 2016:275-287.

[1]	张昆1, 张永伟1, 吴永城1, 张笑文2, 翟世臣2. 基于大模型的设备故障知识图谱自动构建方法[J]. 计算机与现代化, 2024, 0(11): 46-53.
[2]	齐俊1, 2, 曲睿婷2, 教传铭2, 周巧妮2, 郭彦良3, 覃文军3. 基于知识图谱增强大语言模型双碳领域服务[J]. 计算机与现代化, 2024, 0(09): 8-14.
[3]	汪显顺, 熊卿智, 万磊, 李祥, 林重汕, 金安安. 基于知识图谱的区域企业关联可视化及关系挖掘[J]. 计算机与现代化, 2024, 0(08): 11-16.
[4]	杨孟, 杨进, 陈步前. 基于知识图谱的多目标可解释性推荐[J]. 计算机与现代化, 2024, 0(03): 34-40.
[5]	黄政霖, 董宝良. 基于语义和结构增强的时序知识图谱问答方法[J]. 计算机与现代化, 2024, 0(03): 15-23.
[6]	王钰蓥, 王勇. 基于知识图谱和Bi-LSTM的推荐算法[J]. 计算机与现代化, 2021, 0(09): 90-98.
[7]	耿化聪, 梁宏涛, 刘国柱. 基于知识图谱与协同过滤的饮食推荐算法[J]. 计算机与现代化, 2021, 0(08): 24-29.
[8]	张艳, 杨芳, 杨蕾, 韩奎国, 李辉. 基于知识图谱的区块链技术及电力行业应用分析[J]. 计算机与现代化, 2020, 0(12): 55-60.
[9]	张紫璇,陆佳民,姜笑,冯钧. 面向水利信息资源的智能问答系统构建与应用[J]. 计算机与现代化, 2020, 0(03): 65-.
[10]	项威. 事件知识图谱构建技术与应用综述[J]. 计算机与现代化, 2020, 0(01): 10-.
[11]	冯钧，徐新，陆佳民. 水利信息知识图谱的构建与应用[J]. 计算机与现代化, 2019, 0(09): 35-.
[12]	冯钧,柳菁铧,孔盛球. 融合多特征的中文集成实体链接方法[J]. 计算机与现代化, 2019, 0(01): 69-.
[13]	王通,艾中良,张先国. 基于深度学习的威胁情报知识图谱构建技术[J]. 计算机与现代化, 2018, 0(12): 21-.
[14]	申存1,2,3，黄廷磊2,3，梁霄2,3. 基于多粒度特征表示的知识图谱问答[J]. 计算机与现代化, 2018, 0(09): 5-.
[15]	王良萸. 面向碳交易领域的知识图谱构建方法[J]. 计算机与现代化, 2018, 0(08): 114-.