关键词: LoRA微调, 通用语言大模型GLM, 立场检测, 零样本和少样本检测

Abstract:  Stance detection is a key task in natural language processing， which determines the stance of an author based on text analysis. Text stance detection methods transition from early machine learning methods to BERT models， and then evolve to the latest large language models such as ChatGPT. Distinguishing from the closed-source feature of ChatGPT， this paper proposes a text stance detection model， ChatGLM3-LoRA-Stance， by using the domestic open-source ChatGLM3 model. In order to apply large models in professional vertical fields， this paper uses LoRA efficient fine-tuning method. Compared with P-Tuning V2 efficient fine-tuning method， LoRA is more suitable for zero-shot and few-shot text stance detection tasks in text. The paper uses the publicly available VAST dataset to fine-tune the ChatGLM3 model， evaluating the performance of existing models in zero-shot and few-shot scenarios. Experimental results indicate that ChatGLM3-LoRA-Stance model has significantly higher F1 scores than other models on zero-shot and few-shot detection tasks. Therefore， the results verify the potential of large language models on text stance detection tasks， and suggest that that the use of LoRA efficient fine-tuning technology can significantly improve the performance of ChatGLM3 large language model in text stance detection tasks.

Key words:  , LoRA-based fine-tuning, general language large model GLM, stance detection, zero-shot and few-shot detection