Few-Shot Patent Classification Method Integrating Multi-dimensional Prompts and Multi-level Label Expansion

doi:10.3969/j.issn.1006-2475.2025.11.004

Abstract

Abstract: Abstract: To promote the integration of industry， academia， and research and drive the development of emerging and future industries， it is necessary to classify university patents according to industrial needs. However， currently， there is a lack of patent classification resources for emerging industries， and the cost of data annotation is high. Therefore， this paper proposes a few-shot patent classification method that integrates multi-dimensional prompts and multi-level label word expansion for emerging industry patent classification. This method uses BERTopic for topic clustering to obtain the topic keywords in patent texts and uses GLM-4 to extract professional terms from patent texts to help the model understand patents from multiple dimensions at the macro and micro levels. It uses Masked Language Modeling （MLM） and ChatGPT to expand the label word space from multiple levels and provide more abundant and semantically deep label words for the prompt learning model. Experiments are verified on the constructed few-shot patent classification data set and achieve better classification results than baseline models. Moreover， the effect is better than that of the GLM-4 large language model， verifying the effectiveness of the proposed method in few-shot patent classification.

Key words: Key words: prompt learning, prompt engineering, answer engineering, patent classification

CLC Number:

中图分类号：TP391

YOU Xindong, ZHAO Yuxian, LYU Xueqiang, LIU Boshan. Few-Shot Patent Classification Method Integrating Multi-dimensional Prompts and Multi-level Label Expansion[J]. Computer and Modernization, 2025, 0(11): 32-40.

References

［1］ HOSHINO Y， UTSUMI Y， MATSUDA Y， et al. IPC prediction of patent documents using neural network with attention for hierarchical structure［J］. Plos One， 2023，18（3）: e0282361.
［2］ CHIKKAMATH R， PARMAR V R， OTIEFY Y， et al. Patent classification using BERT-for-patents on USPTO［C］// Proceedings of the 2022 5th International Conference on Machine Learning and Natural Language Processing. ACM， 2022: 20-28.
［3］ GROOTENDORST M. BERTopic: Neural topic modeling with a class-based TF-IDF procedure［J］. arXiv preprint arXiv:2203.05794， 2022.
［4］ ZENG A， XU B， WANG B W， et al. ChatGLM: A family of large language models from GLM-130B to GLM-4 All Tools［J］. arXiv preprints arXiv: 2406.12793， 2024.
［5］ CHEN Y L， CHANG Y C. A three-phase method for patent classification［J］. Information Processing & Management， 2012，48（6）:1017-1030.
［6］ LI X， CHEN H H， ZHANG Z， et al. Automatic patent classification using citation network information: An experimental study in nanotechnology［C］// Proceedings of the 7th ACM/IEEE Joint Conference on Digital Libraries. ACM， 2007:419-427.
［7］ ZHANG X Y. Interactive patent classification based on multi-classifier fusion and active learning［J］. Neurocomputing， 2014，127:200-205.
［8］贾杉杉，刘畅，孙连英，等.基于多特征多分类器集成的专利自动分类研究［J］. 数据分析与知识发现， 2017，1（8）:76-84.
［9］包翔，刘桂锋，崔靖华. 多示例多标签学习在中文专利自动分类中的应用研究［J］. 图书情报工作， 2021，65（8）:107-113.
［10］戴佩娟，贺春林，山岳玉蓉. 基于XGBoost的医药专利多标签文本分类［J］. 内江师范学院学报， 2021（10）:55-60.
［11］ CHEN T Q， GUESTRIN C. XGBoost: A scalable tree boosting system［C］// Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining. ACM， 2016:785-794.
［12］ ZHOU P， SHI W， TIAN J， et al. Attention-based bidirectional long short-term memory networks for relation classification［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. ACL，2016: 207-212.
［13］慎金花，陈红艺，张更平，等. 基于层次分类器的专利文本分类模型研究［J］. 情报杂志， 2023，42（8）:157-163.
［14］ LI S B， HU J， CUI Y X， et al. DeepPatent: Patent classification with convolutional neural networks and word embedding［J］. Scientometrics， 2018，117（2）: 721-744.
［15］温超东，曾诚，任俊伟，等. 结合ALBERT和双向门控循环单元的专利文本分类［J］. 计算机应用， 2021，41（2）: 407-412.
［16］ JUNG G， SHIN J H， LEE S J. Impact of preprocessing and word embedding on extreme multi-label patent classification tasks［J］. Applied Intelligence， 2023，53（4）: 4047-4062.
［17］ YOU R H， ZHANG Z H， WANG Z Y， et al. AttentionXML: Label tree-based attention-aware deep model for high-performance extreme multi-label text classification［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. ACM，2019:5820-5830.
［18］徐雪洁，王宝会. 基于文本及历史数据的多标签专利分类算法研究［J/OL］.计算机科学：1-9 （2023-09-26）［2024-01-27］. http://kns.cnki.net/kcms/detail/50.1075.TP.20230925.1335.086.html.
［19］谭晏松，宋铁成. 基于图注意力网络的多标签专利分类方法［J］. 计算机应用与软件， 2023，40（11）:194-200.
［20］ BROWN T B， MANN B， RYDER N， et al. Language models are few-shot learners［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. ACM， 2020，33:1877-1901.
［21］ SCHICK T， SCHÜTZE H. Exploiting cloze questions for few shot text classification and natural language inference［J］. arXiv preprint arXiv:2001.07676， 2020.
［22］ JIANG Z B， XU F F， ARAKI J， et al. How can we know what language models know?［J］. Transactions of the Association for Computational Linguistics， 2020，8: 423-438.
［23］ WALLACE E， FENG S， KANDPAL N， et al. Universal adversarial triggers for attacking and analyzing NLP［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing （EMNLP-IJCNLP）. ACL， 2019:2153-2162.
［24］ LI X L， LIANG P. Prefix-Tuning: Optimizing continuous prompts for generation［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing . ACL， 2021: 4582-4597.
［25］ LIU X， ZHENG Y N， DU Z X， et al. GPT understands， too［J］. AI Open， 2023，5:208-215.
［26］ CUI L Y， WU Y， LIU J， et al. Template-based named entity recognition using BART［C］// Findings of the Association for Computational Linguistics:ACL-IJCNLP 2021.ACL， 2021:1835-1845.
［27］ HU S D， DING N， WANG H D， et al. Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. ACL，2022: 2225-2240.
［28］ SCHICK T， SCHMID H， SCHÜTZE H. Automatically identifying words that can serve as labels for few-shot text classification［C］// Proceedings of the 28th International Conference on Computational Linguistics.ACL，2020: 5569-5578.
［29］ HAMBARDZUMYAN K， KHACHATRIAN H， May J. WARP:Word-level adversarial reprogramming［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL，2021: 4921-4933.
［30］ MCLNNES L， HEALY J， MELVILLE J. UMAP: Uniform manifold approximation and projection for dimension reduction ［J］. arXiv preprint arXiv:1802.03426， 2018.
［31］ MCLNNES L， HEALY J， ASTELS S. HDBSCAN: Hierarchical density based clustering［J］. Journal of Open Source Software， 2017， 2（11）: 205.
［32］ LIAO W X， LIU Z L， DAI H X， et al. Mask-guided BERT for few-shot text classification［J］. Neurocomputing， 2024，610. DOI: 10.1016/j.neucom.2024.128576.
［33］ HU E J， SHEN Y L， WALLIS P， et al. LoRA: Low-rank adaptation of large language models［J］. arXiv preprint arXiv:2106.09685， 2021.

[1]	YANG Huimin, YANG Jin, SUN Yujie. Multi-strategy Dung Beetle Optimization Algorithm for Optimizing UAV Trajectory Planning Problem [J]. Computer and Modernization, 2025, 0(11): 16-31.
[2]	WAN Chengkai1, AN Gaoyun2, CUI Lan3. Crowd Counting Estimation Algorithm of Railway Stations Based on Improved P2PNet [J]. Computer and Modernization, 2025, 0(11): 58-64.
[3]	WANG Jingpeng, CUI Yuyong, CAI Changlin, HE Ming’ao, LI Yinghao, TANG Zhonghe. Improved SOD Algorithm with Cross Modal Interaction and Multi Scale Aggregation [J]. Computer and Modernization, 2025, 0(11): 71-79.
[4]	WAN Siyao1, FANG Zhiwei1, ZHOU Ting1, LYU Hanyun2, LUO Zheng3, CHEN Siyuan1. Optimization and Evaluation of 3D Gaussian Splatting with Dynamic Opacity Filtering [J]. Computer and Modernization, 2025, 0(11): 89-96.
[5]	QIU Xinyu1, CHEN Xiao1, CHEN Mingming2, GAO Hui3, MENG Ziyue3. Fault Diagnosis Method for Charging Pile Based on Vision Transformer [J]. Computer and Modernization, 2025, 0(11): 97-105.
[6]	JIANG Dongmei1, YANG Nuo2 , CHEN Renming2, DONG Changwu3, PENG Chengdong2, 4 . Sublingual Vein Image Segmentation Based on HRNetV2 Model with Variable Kernel Convolution [J]. Computer and Modernization, 2025, 0(10): 14-19.
[7]	ZHOU Bangyuan1, XIN Guojiang1, LIANG Hao2, DING Changsong1. Pancreas Segmentation Based on Two-stage Network of Multiple Attention Mechanisms [J]. Computer and Modernization, 2025, 0(10): 67-72.
[8]	WANG Zhicong1, MA Yihang2, ZHANG Lingxiang3. Oxygen Content Prediction of Circulating Fluidized Bed Boiler Based on THBA-BiLSTM [J]. Computer and Modernization, 2025, 0(10): 73-79.
[9]	WANG Yinbing1, 2, WANG Xingfen1, 3, LI Libo1, 3. Method for Predicting Agricultural Product Prices Based on Improved TimesNet [J]. Computer and Modernization, 2025, 0(10): 89-95.
[10]	XIE Li1, LIU Zhiying2, ZHANG Yu3, QING Chaojin2. Perception and Multimodal Collaborative Risk Assessment Method for Coating Operations [J]. Computer and Modernization, 2025, 0(10): 96-102.
[11]	LYU Tianci, LI Yanhui, CHENG Mengyuan, ZHAO Yuxin, HUANG Chen. Location Data Collection under Local Differential Privacy [J]. Computer and Modernization, 2025, 0(10): 118-126.
[12]	LIU Rongcheng1, XIN Guojiang1, ZHANG Yang1, ZHU Lei2. Natural Environment Tongue Image Segmentation Method Based on Improved Labv3+ Model [J]. Computer and Modernization, 2025, 0(10): 32-36.
[13]	LUO Deyan, XU Yang, ZUO Fengyun, ZHANG Yongdan. Lightweight Garbage Classification and Detection of Improved YOLOv8 [J]. Computer and Modernization, 2025, 0(09): 35-42.
[14]	LIN Ruizi, YAO Da, DAI Xin, SHEN Guoyu, WANG Jiahui, WAN Weiguo. Facial Sketch Image Conversion Based on CycleGAN and Attention Mechanism [J]. Computer and Modernization, 2025, 0(09): 61-66.
[15]	ZHU Lulu, GU Lin. Arbitrary Style Transfer Method with Multi-scale Semantic Adaptation [J]. Computer and Modernization, 2025, 0(09): 73-78.