计算机与现代化 ›› 2023, Vol. 0 ›› Issue (01): 49-57.
出版日期:
2023-03-02
发布日期:
2023-03-02
作者简介:
王浩畅(1974—),女,黑龙江大庆人,教授,博士,研究方向:人工智能,自然语言处理和数据挖掘,E-mail: kinghaosing@gmail.com; 刘如意(1995—),男,江西赣州人,硕士研究生,研究方向:实体关系抽取,E-mail: jsaslry@163.com。
基金资助:
Online:
2023-03-02
Published:
2023-03-02
摘要: 近年来随着深度学习技术的不断革新,预训练模型在自然语言处理中的应用也越来越广泛,关系抽取不再是单纯地依赖传统的流水线方法。预训练语言模型的发展已经极大地推动了关系抽取的相关研究,在很多领域已经超越了传统方法。首先简要介绍关系抽取的发展与经典预训练模型;其次总结当下常用的数据集与评测方法,并分析模型在各数据集上的表现;最后探讨关系抽取发展的挑战与未来研究趋势。
王浩畅, 刘如意. 基于预训练模型的关系抽取研究综述[J]. 计算机与现代化, 2023, 0(01): 49-57.
WANG Hao-chang, LIU Ru-yi. Review of Relation Extraction Based on Pre-training Language Model[J]. Computer and Modernization, 2023, 0(01): 49-57.
[1] | CHINCHOR N, MARSH E. MUC-7 information extraction task definition[C]// Proceedings of the 7th Message Understanding Conference. 1998:359-367. |
[2] | NIST Website. Automatic Content Extraction[EB/OL].[2007-05-28]. http://www.nist.gov/speech/tests/ace/. |
[3] | HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[C]// Proceedings of the 2009 Workshop on Semantic Evaluations: Recent Achievements and Future Directions. 2009:94-99. |
[4] | AONE C, HALVERSON L, HAMPTON T, et al. SRA: Description of the IE2 system used for MUC-7[C]// Proceedings of the 7th Message Understanding Conference. 1998. |
[5] | MNIH A, HINTON G E. A scalable hierarchical distributed language model[C]// Proceedings of the 21st International Conference on Neural Information Processing Systems. 2008:1081-1088. |
[6] | MIKOLOV T, SUTSKEVERI, CHEN K, et al. Distributed representations of words and phrases and their composi-tionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013:3111-3119. |
[7] | PENNINGTON J, SOCHER R, MANNING C. Glove: Global vectors for word representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1532-1543. |
[8] | BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]// Proceedings of 3rd International Conference on Learning Representations. 2015. |
[9] | PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018:2227-2237. |
[10] | RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving 1anguage understanding by generative pretraining[J]. arXiv preprint arXiv:1802.05365, 2018. |
[11] | DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019:4171-4186. |
[12] | YANG Z L, DAI Z H, YANG Y M, et al. XLNET: Generalized autoregressive pretraining for language understanding[C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019:5753-5763. |
[13] | DAI Z H, YANG Z L, YANG Y M, et al. Transformer-XL: Attentive language models beyond a fixed-length context[EB/OL]. arXiv preprint arXiv:1901.02860, 2019. |
[14] | ZHONG Z X, CHEN D Q. A frustratingly easy approach for entity and relation extraction[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021:50-61. |
[15] | XIE C H, LIANG J Q, LIU J P, et al. Revisiting the negative data of distantly supervised relation extraction[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021:3572-3581. |
[16] | SHANG Y M, HUANG H Y, MAo X L. OneRel: Joint entity and relation extraction with one module in one step[J]. Proceedings of the AAAI Conference on Artificial Intelligence. 2022,36(10):11285-11293. |
[17] | SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015:802-810. |
[18] | JOZEFOWICZ R, VINYALS O, SCHUSTER M, et al. Exploring the limits of language modeling[J]. arXiv preprint arXiv:1602.02410, 2016. |
[19] | LE CUN Y, BOSER B, DENKER J S, et al. Handwritten digit recognition with a back-propagation network[C]// Proceedings of the 2nd International Conference on Neural Information Processing Systems. 1989:396-404. |
[20] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017:6000-6010. |
[21] | HOWARD J, RUDER S. Universal language model fine-tuning for text classification[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018:328-339. |
[22] | LIU P J, SALEH M, POT E, et al. Generating Wikipedia by summarizing long sequences[J]. arXiv preprint arXiv:1801.10198, 2018. |
[23] | JIANG X Z, LIANG Y B, CHEN W Z, et al. XLM-K: Improving cross-lingual language model pre-training with multilingual knowledge[J]. arXiv preprint arXiv:2109. 12573, 2021. |
[24] | ZHANG Z Y, HAN X, LIU Z Y, et al. ERNIE: Enhanced language representation with informative entities[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:1441-1451. |
[25] | SUN Y, WANG S H, LI Y K, et al. ERNIE: Enhanced representation through knowledge integration[J]. arXiv preprint arXiv:1904.09223, 2019. |
[26] | LIU X D, HE P C, CHEN W Z, et al. Multi-task deep neural networks for natural language understanding[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:4487-4496. |
[27] | SUN Y, WANG S H, LI Y K, et al. Ernie 2.0: A continual pre-training framework for language understanding[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020,34(5):8968-8975. |
[28] | CUI Y M, CHE W X, LIU T, et al. Pre-training with whole word masking for chinese bert[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021,29:3504-3514. |
[29] | LIU Y H, OTT M, GOYAL N, et al. Roberta: A robustly optimized bert pretraining approach[J]. arXiv preprint arXiv:1907.11692, 2019. |
[30] | JOSHI M, CHEN D Q, LIU Y H, et al. Spanbert: Improving pre-training by representing and predicting spans[J]. Transactions of the Association for Computational Linguistics, 2020,8:64-77. |
[31] | SONG K T, TAN X, QIN T, et al. MASS: Masked sequence to sequence pre-training for language generation[C]// Proceedings of the 36th International Conference on Machine Learning. 2019:5926-5936. |
[32] | LAN Z Z, CHEN M D, GOODMAN S, et al. ALBERT: A lite BERT for self-supervised learning of language representations[J]. arXiv preprint arXiv:1909.11942, 2019. |
[33] | JIAO X Q, YIN Y C, SHANG L F, et al. TinyBERT: Distilling BERT for natural language understanding[C]// Findings of the Association for Computational Linguistics: EMNLP 2020. 2020:4163-4174. |
[34] | SANH V, DEBUT L, CHAUMOND J, et al. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter[J]. arXiv preprint arXiv:1910.01108, 2019. |
[35] | WEI J Q, REN X Z, LI X G, et al. NEZHA: Neural contextualized representation for Chinese language understanding[J]. arXiv preprint arXiv:1909.00204, 2019. |
[36] | PETERS M E, NEUMANN M, LOGAN R, et al. Knowledge enhanced contextual word representations[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019:43-54. |
[37] | WANG J, LU W. Two are better than one: Joint entity and relation extraction with table-sequence encoders[C] // Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:1706-1721. |
[38] | WADDEN D, WENNBERG U, LUAN Y, et al. Entity, relation, and event extraction with contextualized span representations[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019:5784-5789. |
[39] | LUAN Y, WADDEN D, HE L H, et al. A general framework for information extraction using dynamic span graphs[C]// Proceedings of the 2019 Conference of the North. 2019:3036-3046. |
[40] | COHEN A D, ROSENMAN S, GOLDBERG Y. Relation classification as two-way span-prediction[J]. arXiv preprint arXiv:2010.04829, 2020. |
[41] | LI C, TIAN Y. Downstream model design of pre-trained language model for relation extraction task[J]. arXiv preprint arXiv:2004.03786, 2020. |
[42] | TAO Q X, LUO X H, WANG H, et al. Enhancing relation extraction using syntactic indicators and sentential contexts[C]// 2019 IEEE 31st International Conference on Tools with Artificial Intelligence. 2019:1574-1580. |
[43] | WEI Z P, SU J L, WANG Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020:1476-1488. |
[44] | YE H B, ZHANG N Y, DENG S M, et al. Contrastive triple extraction with generative transformer[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(16):14257-14265. |
[45] | ZHOU W X, HUANG K, MA T Y, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(16): 14612-14620. |
[46] | ZENG S, XU R X, CHANG B B, et al. Double graph based reasoning for document-level relation extraction[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:1630-1640. |
[47] | DIXIT K, AL-ONAIZAN Y. Span-level model for relation extraction[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:5308-5314. |
[48] | SANH V, WOLF T, RUDER S. A hierarchical multi-task approach for learning embeddings from semantic tasks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019,33(1):6949-6956. |
[49] | ZHAO Y, WAN H Y, GAO J W, et al. Improving relation classification by entity pair graph[C]// Proceedings of the 11th Asian Conference on Machine Learning. 2019:1156-1171. |
[50] | SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: Distributional similarity for relation learning[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:2895-2905. |
[51] | WU S C, HE Y F. Enriching pre-trained language model with entity information for relation classification[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019:2361-2364. |
[52] | WANG H Y, TAN M, YU M, et al. Extracting multiple-relations in one-pass with pre-trained transformers[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019:1371-1377. |
[53] | ALT C, H[U]BNER M, HENNIG L. Improving relation extraction by pre-trained language representations[C] // Automated Knowledge Base Construction (AKBC). 2018. |
[54] | LUO X K, LIU W J, MA M, et al. BiTT: Bidirectional tree tagging for joint extraction of overlapping entities and relations[J]. arXiv preprint arXiv:2008.13339, 2020. |
[55] | SUN K, ZHANG R C, MENSAH S, et al. Recurrent interaction network for jointly extracting entities and classifying relations[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:3722-3732. |
[56] | YAMADA I, ASAI A, SHINDO H, et al. LUKE: Deep contextualized entity representations with entity-aware self-attention[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:6442-6454. |
[57] | YANG S M, YOO S Y, JEONG O R. DeNERT-KG: Named entity and relation extraction model using DQN, knowledge graph, and BERT[J]. Applied Sciences, 2020,10(18):6429. |
[58] | WANG R Z, TANG D Y, DUAN N, et al. K-Adapter: Infusing knowledge into pre-trained models with adapters[C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021:1405-1418. |
[59] | WANG X Z, GAO T Y, ZHU Z C, et al. KEPLER: A unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021,9(11):176-194. |
[60] | CHEN J, HOEHNDORF R, ELHOSEINY M, et al. Efficient long-distance relation extraction with DG-SpanBERT[J]. arXiv preprint arXiv:2004.03636, 2020. |
[61] | XUE F Z, SUN A X, ZHANG H, et al. GDPNet: Refining Latent multi-view graph for relation extraction[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021,35(16):14194-14202. |
[62] | PENG H, GAO T Y, HAN X, et al. Learning from context or names? An empirical study on neural relation extraction[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:3661-3672. |
[63] | SHI P, LIN J. Simple BERT models for relation extraction and semantic role labeling[J]. arXiv preprint arXiv:1904.05255, 2019. |
[64] | HUANG K, WANG G T, MA T Y, et al. Entity and evidence guided relation extraction for DocRED[J]. arXiv preprint arXiv:2008.12283, 2020. |
[65] | YE D M, LIN Y K, DU J J, et al. Coreferential reasoning learning for language representation[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020:7170-7186. |
[66] | NAN G, GUO Z, SEKULICI, et al. Reasoning with Latent structure refinement for document-level relation extraction[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020:1546-1557. |
[67] | JUNG W, SHIM K. Dual supervision framework for relation extraction with distant supervision and human annotation[C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020:6411-6423. |
[68] | TANG H Z, CAO Y N, ZHANG Z Y, et al. Hin: Hierarchical inference network for document-level relation extraction[C]// Proceedings of the 24th Pacific-Asia Conference, on Knowledge Discovery and Data Mining. 2020:197-209. |
[69] | WANG H, FOCKE C, SYLVESTER R, et al. Fine-tune Bert for DocRED with two-step process[J]. arXiv preprint arXiv:1909.11898, 2019. |
[70] | WANG Y C, YU B W, ZHANG Y Y, et al. TPLinker: Single-stage joint extraction of entities and relations through token pair linking[C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020:1572-1582. |
[1] | 胡崇佳, 刘金洲, 方 立. 基于无监督域适应的室外点云语义分割[J]. 计算机与现代化, 2024, 0(01): 74-79. |
[2] | 林 威. 基于自监督学习和数据回放的新闻推荐模型增量学习方法[J]. 计算机与现代化, 2023, 0(12): 1-6. |
[3] | 梁天恺, 黄康华, 刘凯航, 兰 岚, 曾 碧. 基于双向同态加密的深度联邦图片分类方法[J]. 计算机与现代化, 2023, 0(12): 36-40. |
[4] | 邱凯星, 冯 广. 基于双重特征注意力的多标签图像分类模型[J]. 计算机与现代化, 2023, 0(12): 41-47. |
[5] | 张伯泉, 麦海鹏, 陈嘉敏, 逄锦聚. 基于高灰度值注意力机制的脑白质高信号分割[J]. 计算机与现代化, 2023, 0(12): 67-75. |
[6] | 马泽宇, 叶 宁, 徐 康, 王 甦, 王汝传, . 基于FMCW雷达和ResNeSt-GRU的行为识别方法[J]. 计算机与现代化, 2023, 0(11): 101-107. |
[7] | 李延满, 王必恒, 赵羚焱. 基于轻量化YOLOv5的安全帽检测[J]. 计算机与现代化, 2023, 0(10): 59-64. |
[8] | 黎世达, 项剑文. 一种提高图像识别模型鲁棒性的弱化强化方法[J]. 计算机与现代化, 2023, 0(10): 70-76. |
[9] | 沈加炜, 陆一鸣, 陈晓艺, 钱美玲, 陆卫忠, . 基于深度学习的人体行为检测方法研究综述[J]. 计算机与现代化, 2023, 0(09): 1-9. |
[10] | 刘禅奕, 黄 丹, 薛林雁, 王 涛, 朱 桃, . 改进EfficientNet网络的COVID-19 X光分类[J]. 计算机与现代化, 2023, 0(09): 94-99. |
[11] | 马国祥, 杨凌菲, 严传波, 张志豪, 孙 彬, 王晓荣. 基于深度DenseNet网络的肝包虫病超声影像诊断方法[J]. 计算机与现代化, 2023, 0(09): 100-104. |
[12] | 农皓程, 任德均, 任秋霖, 刘澎笠, 黄德成. 基于改进ConvNeXt的软塑包装表面异常检测算法[J]. 计算机与现代化, 2023, 0(08): 12-17. |
[13] | 欧阳飞, 吴 旭, 向东升. 基于改进YOLOX的垃圾分类检测方法[J]. 计算机与现代化, 2023, 0(08): 68-73. |
[14] | 胡睿杰, 车 逗. 红外小目标检测方法综述[J]. 计算机与现代化, 2023, 0(08): 79-86. |
[15] | 江 蕾, 唐 建, 杨超越, 吕婷婷. 基于CWGAN-GP与CNN的轴承故障诊断方法[J]. 计算机与现代化, 2023, 0(07): 1-6. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||