融合多维提示及多层次标签词扩展的小样本专利分类方法

doi:10.3969/j.issn.1006-2475.2025.11.004

摘要/Abstract

摘要： 摘要：为推动产学研一体化、新兴产业和未来产业发展，高校专利按产业需求进行分类是非常有必要的。然而，当前缺乏面向新兴产业的专利分类资源，而且数据标注成本高昂。因此，本文针对新兴产业专利分类提出一种融合多维提示及多层次标签词扩展的小样本专利分类方法。该方法使用BERTopic进行主题聚类获取专利文本中的主题关键词，使用GLM-4抽取专利文本中的专业术语，帮助模型从宏观和微观层面多维度理解专利；使用Masked Language Modeling（MLM）和ChatGPT从多层次扩充标签词空间，为提示学习模型提供更加丰富且具有语义深度的标签词汇。实验在构建的小样本专利分类数据集上进行验证，均取得了比基线模型更好的分类效果，且效果优于GLM-4大语言模型，验证了所提出的方法在小样本专利分类上的有效性。

关键词: 关键词：提示学习, 提示工程, 答案工程, 专利分类

Abstract: Abstract: To promote the integration of industry， academia， and research and drive the development of emerging and future industries， it is necessary to classify university patents according to industrial needs. However， currently， there is a lack of patent classification resources for emerging industries， and the cost of data annotation is high. Therefore， this paper proposes a few-shot patent classification method that integrates multi-dimensional prompts and multi-level label word expansion for emerging industry patent classification. This method uses BERTopic for topic clustering to obtain the topic keywords in patent texts and uses GLM-4 to extract professional terms from patent texts to help the model understand patents from multiple dimensions at the macro and micro levels. It uses Masked Language Modeling （MLM） and ChatGPT to expand the label word space from multiple levels and provide more abundant and semantically deep label words for the prompt learning model. Experiments are verified on the constructed few-shot patent classification data set and achieve better classification results than baseline models. Moreover， the effect is better than that of the GLM-4 large language model， verifying the effectiveness of the proposed method in few-shot patent classification.

Key words: Key words: prompt learning, prompt engineering, answer engineering, patent classification

中图分类号:

中图分类号：TP391

游新冬, 赵玉贤, 吕学强, 刘勃杉. 融合多维提示及多层次标签词扩展的小样本专利分类方法[J]. 计算机与现代化, 2025, 0(11): 32-40.

YOU Xindong, ZHAO Yuxian, LYU Xueqiang, LIU Boshan. Few-Shot Patent Classification Method Integrating Multi-dimensional Prompts and Multi-level Label Expansion[J]. Computer and Modernization, 2025, 0(11): 32-40.

参考文献

［1］ HOSHINO Y， UTSUMI Y， MATSUDA Y， et al. IPC prediction of patent documents using neural network with attention for hierarchical structure［J］. Plos One， 2023，18（3）: e0282361.
［2］ CHIKKAMATH R， PARMAR V R， OTIEFY Y， et al. Patent classification using BERT-for-patents on USPTO［C］// Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing. ACM， 2022: 20-28.
［3］ GROOTENDORST M. BERTopic: Neural topic modeling with a class-based TF-IDF procedure［J］. arXiv preprint arXiv:2203.05794， 2022.
［4］ ZENG A， XU B， WANG B W， et al. ChatGLM: A family of large language models from GLM-130B to GLM-4 All Tools［J］. arXiv preprints arXiv: 2406.12793， 2024.
［5］ CHEN Y L， CHANG Y C. A three-phase method for patent classification［J］. Information Processing & Management， 2012，48（6）:1017-1030.
［6］ LI X， CHEN H H， ZHANG Z， et al. Automatic patent classification using citation network information: An experimental study in nanotechnology［C］// Proceedings of the 7th ACM/IEEE Joint Conference on Digital Libraries. ACM， 2007:419-427.
［7］ ZHANG X Y. Interactive patent classification based on multi-classifier fusion and active learning［J］. Neurocomputing， 2014，127:200-205.
［8］贾杉杉，刘畅，孙连英，等.基于多特征多分类器集成的专利自动分类研究［J］. 数据分析与知识发现， 2017，1（8）:76-84.
［9］包翔，刘桂锋，崔靖华. 多示例多标签学习在中文专利自动分类中的应用研究［J］. 图书情报工作， 2021，65（8）:107-113.
［10］戴佩娟，贺春林，山岳玉蓉. 基于XGBoost的医药专利多标签文本分类［J］. 内江师范学院学报， 2021（10）:55-60.
［11］ CHEN T Q， GUESTRIN C. XGBoost: A scalable tree boosting system［C］// Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining. ACM， 2016:785-794.
［12］ ZHOU P， SHI W， TIAN J， et al. Attention-based bidirectional long short-term memory networks for relation classification［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. ACL，2016: 207-212.
［13］慎金花，陈红艺，张更平，等. 基于层次分类器的专利文本分类模型研究［J］. 情报杂志， 2023，42（8）:157-163.
［14］ LI S B， HU J， CUI Y X， et al. DeepPatent: Patent classification with convolutional neural networks and word embedding［J］. Scientometrics， 2018，117（2）: 721-744.
［15］温超东，曾诚，任俊伟，等. 结合ALBERT和双向门控循环单元的专利文本分类［J］. 计算机应用， 2021，41（2）: 407-412.
［16］ JUNG G， SHIN J H， LEE S J. Impact of preprocessing and word embedding on extreme multi-label patent classification tasks［J］. Applied Intelligence， 2023，53（4）: 4047-4062.
［17］ YOU R H， ZHANG Z H， WANG Z Y， et al. AttentionXML: Label tree-based attention-aware deep model for high-performance extreme multi-label text classification［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. ACM，2019:5820-5830.
［18］徐雪洁，王宝会. 基于文本及历史数据的多标签专利分类算法研究［J/OL］.计算机科学：1-9 （2023-09-26）［2024-01-27］. http://kns.cnki.net/kcms/detail/50.1075.TP.20230925.1335.086.html.
［19］谭晏松，宋铁成. 基于图注意力网络的多标签专利分类方法［J］. 计算机应用与软件， 2023，40（11）:194-200.
［20］ BROWN T B， MANN B， RYDER N， et al. Language models are few-shot learners［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. ACM， 2020，33:1877-1901.
［21］ SCHICK T， SCHÜTZE H. Exploiting cloze questions for few shot text classification and natural language inference［J］. arXiv preprint arXiv:2001.07676， 2020.
［22］ JIANG Z B， XU F F， ARAKI J， et al. How can we know what language models know?［J］. Transactions of the Association for Computational Linguistics， 2020，8: 423-438.
［23］ WALLACE E， FENG S， KANDPAL N， et al. Universal adversarial triggers for attacking and analyzing NLP［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. ACL， 2019:2153-2162.
［24］ LI X L， LIANG P. Prefix-Tuning: Optimizing continuous prompts for generation［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing . ACL， 2021: 4582-4597.
［25］ LIU X， ZHENG Y N， DU Z X， et al. GPT understands， too［J］. AI Open， 2023，5:208-215.
［26］ CUI L Y， WU Y， LIU J， et al. Template-based named entity recognition using BART［C］// Findings of the Association for Computational Linguistics:ACL-IJCNLP 2021.ACL， 2021:1835-1845.
［27］ HU S D， DING N， WANG H D， et al. Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. ACL，2022: 2225-2240.
［28］ SCHICK T， SCHMID H， SCHÜTZE H. Automatically identifying words that can serve as labels for few-shot text classification［C］// Proceedings of the 28th International Conference on Computational Linguistics.ACL，2020: 5569-5578.
［29］ HAMBARDZUMYAN K， KHACHATRIAN H， May J. WARP:Word-level adversarial reprogramming［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL，2021: 4921-4933.
［30］ MCLNNES L， HEALY J， MELVILLE J. UMAP: Uniform manifold approximation and projection for dimension reduction ［J］. arXiv preprint arXiv:1802.03426， 2018.
［31］ MCLNNES L， HEALY J， ASTELS S. HDBSCAN: Hierarchical density based clustering［J］. Journal of Open Source Software， 2017， 2（11）: 205.
［32］ LIAO W X， LIU Z L， DAI H X， et al. Mask-guided BERT for few-shot text classification［J］. Neurocomputing， 2024，610. DOI: 10.1016/j.neucom.2024.128576.
［33］ HU E J， SHEN Y L， WALLIS P， et al. LoRA: Low-rank adaptation of large language models［J］. arXiv preprint arXiv:2106.09685， 2021.

[1]	杨会敏, 杨进, 孙雨婕. 多策略蜣螂优化算法优化无人机航迹规划问题[J]. 计算机与现代化, 2025, 0(11): 16-31.
[2]	万成凯1, 安高云2, 崔岚3. 基于改进P2PNet的铁路车站人群数量估计算法[J]. 计算机与现代化, 2025, 0(11): 58-64.
[3]	王晶鹏, 崔雨勇, 蔡长霖, 赫明骜, 李颖浩, 唐中和. 一种改进的跨模态交互和多尺度聚合的SOD算法[J]. 计算机与现代化, 2025, 0(11): 71-79.
[4]	万思瑶1, 方智威1, 周婷1, 吕晗韵2, 罗正3, 陈思源1. 动态不透明度滤波的三维高斯溅射优化与评估[J]. 计算机与现代化, 2025, 0(11): 89-96.
[5]	仇新宇1, 陈霄1, 陈铭明2, 高辉3, 孟子悦3. 基于视觉Transformer的充电桩故障诊断方法[J]. 计算机与现代化, 2025, 0(11): 97-105.
[6]	蒋冬梅1, 杨诺2, 陈仁明2, 董昌武3, 彭成东2, 4. 基于变核卷积的HRNetV2模型对舌下络脉图像分割算法[J]. 计算机与现代化, 2025, 0(10): 14-19.
[7]	周邦湲1, 辛国江1, 梁昊2, 丁长松1. 基于两阶段的多重注意力机制网络的胰腺分割[J]. 计算机与现代化, 2025, 0(10): 67-72.
[8]	王智聪1, 马祎航2, 张凌祥3. 基于THBA-BiLSTM循环流化床锅炉含氧量预测[J]. 计算机与现代化, 2025, 0(10): 73-79.
[9]	王饮冰1, 2, 王兴芬1, 3, 李立博1, 3. 基于改进TimesNet模型的农产品价格预测方法[J]. 计算机与现代化, 2025, 0(10): 89-95.
[10]	谢礼1, 刘芝樱2, 张瑜3, 卿朝进2. 感知与多模态协同的涂覆作业风险评估方法[J]. 计算机与现代化, 2025, 0(10): 96-102.
[11]	吕天赐, 李艳辉, 成梦圆, 赵玉鑫, 黄臣. 基于本地差分隐私的位置数据收集方法[J]. 计算机与现代化, 2025, 0(10): 118-126.
[12]	刘嵘澂1, 辛国江1, 张杨1, 朱磊2. 基于改进的DeepLabv3+模型的自然环境下舌象分割方法[J]. 计算机与现代化, 2025, 0(10): 32-36.
[13]	罗德艳, 徐杨, 左锋云, 张永丹. 改进YOLOv8的轻量化垃圾分类检测[J]. 计算机与现代化, 2025, 0(09): 35-42.
[14]	林睿姿, 姚达, 戴欣, 沈国誉, 王嘉慧, 万伟国. 基于CycleGAN和注意力机制的人脸素描图像转换[J]. 计算机与现代化, 2025, 0(09): 61-66.
[15]	祝露露, 谷林. 基于多尺度语义调整的任意风格迁移方法[J]. 计算机与现代化, 2025, 0(09): 73-78.