Tibetan Medical Entity Recognition Based on Tibetan BERT

Abstract

Abstract: Tibetan medicine character embedding is of great significance for Tibetan medical entity recognition， but there is a lack of high-quality Tibetan language model. Combined with Tibetan structural characteristics， the BERT model based on syllable is trained by using ordinary Tibetan news text, and a BERT-BiLSTM-CRF model is built by using the Tibetan BERT model. Firstly， the model uses Tibetan BERT model to learn the character embedding of Tibetan medicine text, and enhances the ability of character embedding to express Tibetan characters and their context information. And then， the BiLSTM layer is used to further extract the dependencies between characters in Tibetan medicine text. Finally， the CRF layer is used to strengthen the legitimacy of the label sequence. The experimental results show that using Tibetan BERT model to initialize character embedding is helpful to improve the recognition of Tibetan medical entity， and the F1 value reaches 96.18%.

Key words: Tibetan, Tibetan medicine, NER, BERT, BiLSTM

ZHU Ya-jun, Yong Tso, Nyima Tashi, . Tibetan Medical Entity Recognition Based on Tibetan BERT[J]. Computer and Modernization, 2023, 0(01): 43-48.

References

［1］高艳菊，张丹. 当代中国藏医药学主流学术群体及其代表人物［J］. 世界科学技术-中医药现代化， 2020，22（8）:2927-2935.
［2］格知加，尼玛次仁. 藏医特色疗法治疗真布病（风湿性关节炎）研究进展［J］. 亚太传统医药， 2021，17（11）:27-29.
［3］肖宗朋，刘鸿，陈建英，等. 基于数据挖掘的疫病藏医用药规律研究［J］. 中国民族民间医药， 2021，30（4）:1-4.
［4］德央，尼玛玉珍. 浅谈藏药经典配方中矿物药的用量及现状［J］. 西藏科技， 2021（1）:60-61.
［5］冯学梅，雅才华，毕文忠，等. 治疗类风湿病的藏医药浴水中生物元素的因子分析［J］. 中国高原医学与生物学杂志， 2021，42（4）:265-268.
［6］张佛晓. 数据挖掘技术在藏医中的研究综述［J］. 数码世界， 2019（6）:53-54.
［7］陈瑶，葛卫红，廖俊. 深度学习在医药领域命名实体识别中的研究进展［J］. 药学进展， 2020，44（1）:28-34.
［8］ LAFFERTY J， MCCALLUM A， PEREIRA F. Conditional random fields: Probabilistic models for segmenting and labeling sequence data［C］// Proceedings of the 18th International Conference on Machine Learning. 2001:282-289.
［9］邵诗韵，周宇，杨蕾，等. 基于条件随机场的电力工程标书文本实体识别方法［J］. 计算机与现代化， 2020（12）:72-77.
［10］张磊. 特定领域的命名实体识别方法的研究［J］. 计算机与现代化， 2018（3）:60-64.
［11］万静，涂喆，冯晓. 基于条件随机场的医药领域症状信息抽取［J］. 北京化工大学学报（自然科学版）， 2016，43（1）:98-103.
［12］高佳奕，刘震，杨涛，等. 基于条件随机场的中医临床医案症状命名实体抽取研究［J］. 世界科学技术-中医药现代化， 2020，22（6）:1947-1954.
［13］ HUANG Z H， XU W， YU K. Bidirectional LSTM-CRF models for sequence tagging［J］. arXiv preprint arXiv:1508.01991， 2015.
［14］ HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997，9（8）:1735-1780.
［15］ GAJENDRAN S， MANJULA D， SUGUMARAN V. Character level and word level embedding with bidirectional LSTM-
Dynamic recurrent neural network for biomedical named entity recognition from literature［J］. Journal of Biomedical Informatics， 2020，112. OI:10.1016/j.jbi.2020.103609.
［16］ CHO M， HA J， PARK C， et al. Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition［J］. Journal of Biomedical Informatics，2020，103. DOI: 10.1016/j.jbi.2020.103381.
［17］殷章志，李欣子，黄德根，等. 融合字词模型的中文命名实体识别研究［J］. 中文信息学报， 2019，33（11）:95-100.
［18］冯云霞，衣鹏，韩正亮，等. 基于CNN-BGRU-CRF的中文电子病历实体抽取方法［J］. 计算机与现代化， 2020 （11）:60-64.
［19］张艺品，关贝，吕荫润，等. 深度学习基础上的中医实体抽取方法研究［J］. 医学信息学杂志， 2019，40（2）:58-63.
［20］高甦，金佩，张德政. 基于深度学习的中医典籍命名实体识别研究［J］. 情报工程， 2019，5（1）:113-123.
［21］牧杨子. 基于半监督学习的中文电子病历实体识别和实体关系抽取研究［D］. 海口:海南大学， 2018.
［22］何家欢，刘勇国，蒋羽，等. 藏药药理命名实体识别［J］. 医学信息学杂志， 2020，41（4）:30-36.
［23］王银瑞，彭敦陆，陈章，等. Trans-NER:一种迁移学习支持下的中文命名实体识别模型［J］. 小型微型计算机系统， 2019，40（8）:1622-1626.
［24］朱岩，张利，王煜. 基于RoBERTa-WWM的中文电子病历命名实体识别［J］. 计算机与现代化， 2021（2）:51-55.
［25］景慎旗，赵又霖. 面向中文电子病历文书的医学命名实体识别研究——一种基于半监督深度学习的方法［J］. 信息资源管理学报， 2021，11（6）:105-115.
［26］ DEVLIN J， CHANG M W， LEE K， et al. BERT: Pre-training of deep bidirectional transformers for language understanding［J］. arXiv preprint arXiv:1810.04805， 2018.
［27］ LAMPLE G， BALLESTEROS M， SUBRAMANIAN S， et al. Neural architectures for named entity recognition［C］// Proceedings of 2016 North American Chapter of the Association for Computational Linguistics-Human Language Technologies. 2016:260-270.
［28］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:6000-6010.
［29］丹曲. 简述藏医学名著《四部医典》及其影响［J］. 中央民族学院学报（哲学社会科学版）， 1987（6）:88-91.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[3]	GONG Yicheng1, 2, LIU Qing1, 2. Beijing Opera Binary Classification Based on RF-LCE-BiLSTM-Attention-AMSSA Model [J]. Computer and Modernization, 2024, 0(11): 7-12.
[4]	LI Junchao1, YOU Fei1, ZHANG Chao2, SU Lele2, GONG Yan2 . BiLSTM-Attention Prediction Model and Error Analysis #br# Based on Novel Multi-objective Coati Optimization Algorithm [J]. Computer and Modernization, 2024, 0(11): 70-76.
[5]	MA Yu, YANG Yong, REN Ge, Palidan Tuerxun. Automated Essay Scoring Method Based on GCN and Fine Tuned BERT [J]. Computer and Modernization, 2024, 0(09): 33-37.
[6]	ZHAO Dun1, SHE Xuebing2, WU Changxing3. Named Entity Recognition in Field of Party Building Based on BERT-BiLSTM-CRF [J]. Computer and Modernization, 2024, 0(09): 91-94.
[7]	XU Xin’ai, LI Gang. An Image Generation Method of Classroom Expression Images [J]. Computer and Modernization, 2024, 0(08): 88-91.
[8]	WANG Zhiqiang, ZHENG Shuang. Semi-supervised Image Generation Model Based on StyleGAN [J]. Computer and Modernization, 2024, 0(06): 14-18.
[9]	Renqingzhuoma1, 2, 3, Yongcuo1, 2, 3, TANG Chaochao1, 2, 3. Data Filtering Strategies for Tibetan-Chinese Neural Machine Translation [J]. Computer and Modernization, 2024, 0(06): 19-24.
[10]	WANG Jia1, ZHANG Yunlong1, JU Weigang1, ZHOU Zhipeng2, MI Chuanmin2. A General Platform for Energy Saving and Consumption Reduction of Server-class#br# Environmental Resources [J]. Computer and Modernization, 2024, 0(05): 61-68.
[11]	LI Jiaduo, YAN Xiuying. Method of Location and Capacity Determination for Distributed Generator Based on#br# Enhanced Capuchin Search Algorithm [J]. Computer and Modernization, 2024, 0(04): 27-32.
[12]	XIAO Mengsi1, WU Jianbin1, TU Yameng1, YUAN Linfeng2. GAN-generated Fake Images Recognition Based on Improved ConvNeXt [J]. Computer and Modernization, 2024, 0(04): 38-42.
[13]	YU Lihui, HU Shaowen, HUANG Langxin, LUO Shuhuan. A Source Code Security Vulnerability Detection Method Using ChatGPT [J]. Computer and Modernization, 2024, 0(04): 88-91.
[14]	LU Zi-han1, ZHANG Dong1, YANG Yan1, YANG Shuang2. Breast Cancer Immunohistochemical Image Generation Based on Generative Adversarial Network [J]. Computer and Modernization, 2024, 0(03): 92-96.
[15]	LIU Yan-hong, YANG Qiu-xiang. Image Dehazing Algorithm with Improved Generative Adversarial Network [J]. Computer and Modernization, 2024, 0(02): 56-63.