Named Entity Recognition in Electronic Medical Record Based on BERT

doi:10.3969/j.issn.1006-2475.2024.01.014

Abstract

Abstract: Abstract：Electronic medical record is an important resource for the preservation， management and transmission of patients’medical records. It is also an important text record for doctors’ diagnosis and treatment of diseases. Through the electronic medical record named entity recognition （NER） technology， diagnosis and treatment information such as symptoms， diseases and drug names can be extracted from the electronic medical record efficiently and intelligently. It is helpful for structured electronic medical records to use machine learning and other technologies for diagnosis and treatment regularity mining. In order to efficiently identify named entities in electronic medical records， a named entity recognition method based on BERT and bidirectional long short-term memory network （BILSTM） with fusion adversarial training （FGM） is proposed， referred to as BERT-BILSTM-CRF-FGM （BBCF）. After preprocessing by correcting the Chinese electronic medical record corpus provided by the 2017 National Knowledge Graph and Semantic Computing Conference （CCKS2017）， the BERT-BILSTM-CRF-FGM model is used to recognize five types of entities in the corpus， with an average F1 score of 92.84%. Compared to the BERT model based on the inflated convolutional neural network （BERT-IDCNN-CRF） and the conditional random field model based on BILSTM （BILSTM-CRF）， the proposed method has higher F1 score and faster convergence speed， which can more efficiently structure electronic medical record text.

Key words: Key words：electronic medical record, named entity recognition, BERT, FGM, BILSTM （Bidirectional Long Short-Term Memory Network）, conditional random field

CLC Number:

TP391.1

ZHENG Li-rui, XIAO Xiao-xia, ZOU Bei-ji, LIU Bin, ZHOU Zhan. Named Entity Recognition in Electronic Medical Record Based on BERT[J]. Computer and Modernization, 2024, 0(01): 87-91.

References

［1］冀振燕，孔德焱，刘伟，等. 基于深度学习的命名实体识别研究［J］. 计算机集成制造系统， 2022，6:1603-1615.
［2］李冬梅，罗斯斯，张小平，等. 命名实体识别方法研究综述［J/OL］. 计算机科学与探索， 2022，16（9）:1954-1968.
［3］刘浏，王东波. 命名实体识别研究综述［J］. 情报学报， 2018，37（3）:329-340.
［4］吴智妍，金卫，岳路，等. 电子病历命名实体识别技术研究综述［J］. 计算机工程与应用， 2022，58（21）:13-29.
［5］杨飞洪，张宇，覃露，等.中文电子病历的命名实体识别研究进展［J］.中国数字医学， 2020，15（2）:9-12.
［6］夏宇彬，郑建立，赵逸凡，等.基于深度学习的电子病历命名实体识别［J］.电子科技， 2018，31（11）:31-34.
［7］欧阳恩，李作高，李昱熙，等.基于深度学习的电子病历命名实体识别及其在知识发现中的应用［J］.中国卫生信息管理杂志， 2018，15（4）:469-473.
［8］江千军，桂前进，王磊，等.命名实体识别技术研究进展综述［J］.电力信息与通信技术， 2022，20（2）:15-24.
［9］ DEVLIN J， CHANG M W， LEE K， et al. BERT: Pretraining of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:4171-4186.
［10］吴小雪，张庆辉.预训练语言模型在中文电子病历命名实体识别上的应用［J］.电子质量， 2020（9）:61-65.
［11］何涛，陈剑，闻英友.基于BERT-CRF模型的电子病历实体识别研究［J］.计算机与数字工程， 2022，50（3）:639-643.
［12］梁文桐，朱艳辉，詹飞，等.基于BERT的医疗电子病历命名实体识别［J］.湖南工业大学学报， 2020，34（4）:54-62.
［13］李妮，关焕梅，杨飘，等.基于BERT-IDCNN-CRF的中文命名实体识别方法［J］.山东大学学报（理学版）， 2020，55（1）:102-109.
［14］ HU S L， ZHANG H J， HU X S， et al. Chinese named entity recognition based on BERT-CRF model［C］// 2022 IEEE/ACIS 22nd International Conference on Computer and Information Science （ICIS）. 2022:105-108.
［15］ MAO J H， LIU W L. Hadoken: A BERT-CRF model for medical document anonymization［C］// Proceedings of the Iberian Languages Evaluation Forum Co-located with 35th Conference of the Spanish Society for Natural Language Processing. 2019:720-726.
［16］ CAI X C， SUN E H， LEI J L. Research on application of named entity recognition of electronic medical records based on BERT-IDCNN-CRF model［C］// Proceedings of the 6th International Conference on Graphics and Signal Processing. 2022:80-85.
［17］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:6000-6010.
［18］王若佳，魏思仪，王继民. BiLSTM-CRF模型在中文电子病历命名实体识别中的应用研究［J］. 文献与数据学报，2019，1（2）:53-66.
［19］谢腾，杨俊安，刘辉. 基于BERT-BiLSTM-CRF模型的中文实体识别［J］. 计算机系统应用， 2020，29（7）:48-55.
［20］ LAFFERTY J D， MCCALLUM A， PEREIRA F C N， et al. Conditional random fields: Probabilistic models for segmenting and labeling sequence data［C］// Proceedings of the 18th International Conference on Machine Learning. 2001:282-289.
［21］ MIYATO T， DAI A M， GOODFELLOW I. Adversarial training methods for semi-supervised text classification［J］. arXiv preprint arXiv:1605.07725， 2016.
［22］赵辉，庞海婷，冯珊珊，等. 中文命名实体识别技术综述［J］. 长春工业大学学报（自然科学版）， 2021，42（5）:444-450.
［23］李妮，关焕梅，杨飘，等. 基于BERT-IDCNN-CRF的中文命名实体识别方法［J］. 山东大学学报（理学版）， 2020，55（1）:102-109.
［24］陈廷寅，冯嵩. 基于IDCNN+CRF和注意力机制的电子病历命名实体识别方法及模型稳定性研究［J］. 中国数字医学， 2022，17（11）:1-5.
［25］万里，罗曜儒，李智，等. 基于字词联合训练的Bi-LSTM中文电子病历命名实体识别［J］. 中国数字医学，2019，14（2）:54-56.
［26］李纲，潘荣清，毛进，等. 整合BiLSTM-CRF网络和词典资源的中文电子病历实体识别［J］. 现代情报， 2020，40（4）:3-12.
［27］李博，康晓东，张华丽，等. 采用Transformer-CRF的中文电子病历命名实体识别［J］. 计算机工程与应用， 2020，56（5）:153-159.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	MA Yu, YANG Yong, REN Ge, Palidan Tuerxun. Automated Essay Scoring Method Based on GCN and Fine Tuned BERT [J]. Computer and Modernization, 2024, 0(09): 33-37.
[3]	ZHAO Dun1, SHE Xuebing2, WU Changxing3. Named Entity Recognition in Field of Party Building Based on BERT-BiLSTM-CRF [J]. Computer and Modernization, 2024, 0(09): 91-94.
[4]	QIAO Lu, SUN You-chao, WU Hong-lan. Information Extraction for Aircraft Fault Text [J]. Computer and Modernization, 2024, 0(03): 61-66.
[5]	WANG Zi-chen, QU You-li. A Partition Inverted Index Compression Algorithm Based on CRF [J]. Computer and Modernization, 2024, 0(02): 36-42.
[6]	CUI Shao-guo, HU Guang-ping. Nested Named Entity Recognition Based on Semantic Segmentation [J]. Computer and Modernization, 2024, 0(02): 69-74.
[7]	LIU Yu-peng, GE Yan, DU Jun-wei, CHEN Zhuo. Joint Extraction Method of Entities and Relations Based on FGM and Pointer Annotation [J]. Computer and Modernization, 2023, 0(11): 1-5.
[8]	TANG Shi-qi, ZHOU Rui-ping, XIE Shi-bin, LIU Meng-chi, XIAO Wen, . Cross-language Multi-label Sentiment Classification Based on Stacked Denoising AutoEncoder [J]. Computer and Modernization, 2023, 0(11): 6-12.
[9]	LI Shi-yue, MENG Jia-na, YU Yu-hai, LI Xue-ying, XU Ying-ao. Aspect Based Sentiment Analysis Model Based on Knowledge Enhancement [J]. Computer and Modernization, 2023, 0(10): 1-8.
[10]	WANG Hong-jie, XU Sheng-chao. Clustering Method of Cloud Platform Abnormal Transmission Data Based on Hilbert Similarity [J]. Computer and Modernization, 2023, 0(09): 27-31.
[11]	YU Tao, ZHANG Ying, Yong Tso, . Tibetan Named Entity Recognition Based on Small Sample Learning [J]. Computer and Modernization, 2023, 0(05): 13-19.
[12]	XIE Shi-chao, HUANG Wei, REN Xiang-hui. A Text Entity Linking Method Based on BERT [J]. Computer and Modernization, 2023, 0(02): 58-61.
[13]	ZHU Ya-jun, Yong Tso, Nyima Tashi, . Tibetan Medical Entity Recognition Based on Tibetan BERT [J]. Computer and Modernization, 2023, 0(01): 43-48.
[14]	YU Qing, MA Zhi-long, XU Chun. Medical Knowledge Extraction Based on BERT and Non-autoregressive [J]. Computer and Modernization, 2023, 0(01): 120-126.
[15]	HUANG Zhong-xiang, LI Ming. Text Classification Based on ALBERT Combined with Bidirectional Network [J]. Computer and Modernization, 2022, 0(10): 8-12.