基于BERT和非自回归的医疗知识抽取

摘要/Abstract

摘要： 为避免实体与关系独立抽取产生的误差累计及实体重叠问题，提出一种基于BERT和非自回归的联合抽取模型来进行医疗知识抽取。首先，通过BERT预训练语言模型进行句子编码；然后，采用非自回归（NAR， Non-autoregressive）的方法实现并行解码，抽取关系类型，并根据头尾实体的位置索引抽取实体，得到医疗实体的关系三元组；最后，将抽取出的实体和关系导入Neo4j图数据库中实现知识可视化。通过对电子病历中的数据进行人工标注得到数据集，实验结果表明，基于BERT和非自回归联合学习模型的F1值为0.92，precision值为0.93，recall值为0.92，与现有模型相比3项评价指标均有提升，表明本文方法能够有效抽取电子病历中的医疗知识。

关键词: 联合学习, 非自回归, BERT, 实体重叠, 电子病历

Abstract: In order to avoid the problems of error accumulation and entity overlap caused by the pipeline entity relation extraction model， a joint extraction model based on BERT and Non-autoregressive is established for medical knowledge extraction. Firstly， with the help of the BERT pre-trained language model， the sentence code is obtained. Secondly， the Non-autoregressive method is proposed to achieve parallel decoding， extract the relationship type， extract entities according to the index of the subject and object entities， and obtain the medical triplet. Finally， we import the extracted triples into the Neo4j graph database and realize knowledge visualization. The dataset is derived from manual labeling of data in electronic medical records. The experimental results show that the F1 value， precision and recall based on BERT and non-autoregressive joint learning model are 0.92， 0.93 and 0.92， respectively. Compared with the existing model， the three evaluation indicators have been improved， indicating that the proposed method can effectively extract medical knowledge from electronic medical records.

Key words: joint learning, non-autoregressive, BERT, entity overlap, electronic medical record

于清, 马志龙, 徐春. 基于BERT和非自回归的医疗知识抽取[J]. 计算机与现代化, 2023, 0(01): 120-126.

YU Qing, MA Zhi-long, XU Chun. Medical Knowledge Extraction Based on BERT and Non-autoregressive[J]. Computer and Modernization, 2023, 0(01): 120-126.

参考文献

［1］宁尚明，滕飞，李天瑞. 基于多通道自注意力机制的电子病历实体关系抽取［J］. 计算机学报， 2020，43（5）:916-929.
［2］贾辛洪，宋文爱，李伟岩，等. 慢阻肺知识图谱的构建研究与实现［J］. 小型微型计算机系统， 2020，41（7）:1371-1374.
［3］贾丽娜，陈恒，李冠宇. 基于注意力混合模型的中文医疗问答匹配［J］. 计算机应用与软件， 2021，38（11）:148-154.
［4］杜优. 基于信息融合的医疗影像辅助决策研究［J］. 智能计算机与应用， 2019，9（2）:91-95.
［5］ SOCHER R，HUVAL B，MANNING C D，et al. Semantic compositionality through recursive matrix-vector spaces ［C］// Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012:1201-1211.
［6］ ZENG D J， LIU K，LAI S W， et al.Relation classification via convolutional deep neural network ［C］// Proceedings of the 25th International Conference on Computational Linguistics: Technical Paper. 2014: 2335-2344.
［7］冯云霞，衣鹏，韩正亮，等. 基于CNN-BGRU-CRF的中文电子病历实体抽取方法［J］. 计算机与现代化， 2020（11）:60-64.
［8］ CHIKKA V R， KARLAPALEM K. A hybrid deep learning approach for medical relation extraction［J］. arXiv preprint arXiv: 1806.11189， 2018.
［9］ UZUNER O， MAILOA J， RYAN R， et al. Semantic relations for problem-oriented medical records［J］. Artificial Intelligence in Medicine， 2010，50（2）: 63-73.
［10］ SUNIL K S，ASHISH A，KRISHNADEV O，et al. Relation extraction from clinical texts using domain invariant convolutional neural network［C］// Workshop on Biomedical Natural Language Processing. 2016:206-215.
［11］ DEVLIN J，CHANG M W，LEE K，et al. BERT：Pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019，1: 4171-4186.
［12］ ZHANG X H，ZHANG Y Y，ZHANG Q，et al. Extracting comprehensive clinical information for breast cancer using deep learning methods［J］. International Journal of Medical Informatics， 2019，132:103985.1-103985.7.
［13］ CHRISTOPOULOU F， TRAN T， SAHU S K， et al. Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods［J］. Journal of the American Medical Informatics Association， 2020，27（1）:39-46.
［14］武小平，张强，赵芳，等. 基于BERT的心血管医疗指南实体关系抽取方法［J］. 计算机应用， 2021，41（1）:145-149.
［15］李丽双，袁光辉，刘晗喆. 基于位置降噪和丰富语义的电子病历实体关系抽取［J］. 中文信息学报， 2021，35（8）:89-97.
［16］张玉坤，刘茂福，胡慧君. 基于联合神经网络模型的中文医疗实体分类与关系抽取［J］. 计算机工程与科学， 2019，41（6）:1110-1118.
［17］ GETOOR L，TASKAR B. Global inference for entity and relation identification via a linear programming formulation［C］// Introduction to Statistical Relational Learning， 2007:553-580.
［18］ MIWA M， BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016，1:1105-1116.
［19］ ZHENG S C， WANG F BAO H Y， et al. Joint extraction of entities and relations based on a novel tagging scheme［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017:1227-1236.
［20］ XU J，LI Z H，WEI Q，et al. Applying a deep learning-based sequence labeling approach to detect attributes of medical concepts in clinical text［J］. BMC Medical Informatics and Decision Making， 2019，19（S5）: Article Number 236.
［21］ PARSAEIMEHR E，FARTASH M，TORKESTANI J A. An enhanced deep neural network-based architecture for joint extraction of entity mentions and relations［J］. International Journal Of Fuzzy Logic And Intelligent Systems，2020，20（1）: 69-76.
［22］ WEI Z P，SU J L，WANG Y，et al. A novel cascade binary tagging framework for relational triple extraction［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020:1476-1488.
［23］ WANG Y C，YU B W，ZHANG Y Y，et al. TPLinker: Single-stage joint extraction of entities and relations through token pair linking［C］// The 28th International Conference on Computational Linguistics. 2020:1572-1582.
［24］ ZENG X R，ZENG D J，HE S Z，et al. Extracting relational facts by an end-to-end neural model with copy mechanism［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018:506-514.
［25］ CABOT P H，NAVIGLI R.REBEL：Relation extraction by end-to-end language generation［C］// Findings of the Association for Computational Linguistics：EMNLP 2021. 2021：2370-2381.
［26］贾浩，王煦，季佰军，等. 基于掩码机制的非自回归神经机器翻译［J］. 厦门大学学报（自然科学版）， 2021，60（4）:648-654.
［27］邓建国，张素兰，张继福，等. 监督学习中的损失函数及应用研究［J］. 大数据， 2020，6（1）:60-80.
［28］ NAYAK T，NG H T. Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction [C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020:8528-8535.

[1]	郑久超, 赵新元. 基于主题与描述信息的实体链接方法[J]. 计算机与现代化, 2024, 0(12): 10-14.
[2]	马钰, 杨勇, 任鸽, 帕力旦·吐尔逊. 基于GCN和微调BERT的作文自动评分方法[J]. 计算机与现代化, 2024, 0(09): 33-37.
[3]	赵盾1, 佘学兵2, 邬昌兴3. 基于BERT-BiLSTM-CRF党建领域命名实体识别[J]. 计算机与现代化, 2024, 0(09): 91-94.
[4]	王谭, 陈金广, 马丽丽. 融合词典信息和句子语义的中文命名实体识别[J]. 计算机与现代化, 2024, 0(03): 24-28.
[5]	郑立瑞, 肖晓霞, 邹北骥, 刘彬, 周展. 基于BERT的电子病历命名实体识别[J]. 计算机与现代化, 2024, 0(01): 87-91.
[6]	刘玉鹏, 葛艳, 杜军威, 陈卓. 融合FGM和指针标注的实体关系联合抽取方法[J]. 计算机与现代化, 2023, 0(11): 1-5.
[7]	唐诗琪, 周瑞平, 谢仕斌, 刘梦赤, 肖文, . 基于栈式降噪编码器的跨语言多标签情感分类[J]. 计算机与现代化, 2023, 0(11): 6-12.
[8]	李诗月, 孟佳娜, 于玉海, 李雪莹, 许英傲. 基于知识增强的方面级情感分析方法[J]. 计算机与现代化, 2023, 0(10): 1-8.
[9]	谢世超, 黄蔚, 任祥辉. 一种基于BERT的文本实体链接方法[J]. 计算机与现代化, 2023, 0(02): 58-61.
[10]	朱亚军, 拥措, 尼玛扎西, . 基于藏文BERT的藏医药医学实体识别[J]. 计算机与现代化, 2023, 0(01): 43-48.
[11]	黄忠祥, 李明. ALBERT结合双向网络的文本分类[J]. 计算机与现代化, 2022, 0(10): 8-12.
[12]	张军, 邱龙龙. 一种基于BERT和池化操作的文本分类模型[J]. 计算机与现代化, 2022, 0(06): 1-7.
[13]	陈钢. 融合RoBERTa和特征提取的政务热线工单分类[J]. 计算机与现代化, 2022, 0(06): 21-26.
[14]	樊海玮, 秦佳杰, 孙欢, 张丽苗, 鲁芯丝雨. 基于BERT与BiGRU-CRF的交通事故文本信息提取模型[J]. 计算机与现代化, 2022, 0(05): 10-15.
[15]	刘梦颖, 王勇. 基于文本双表示模型的微博热点话题发现[J]. 计算机与现代化, 2021, 0(12): 110-115.