基于动态卷积和自注意力的序列推荐模型

doi:10.3969/j.issn.1006-2475.2023.10.002

摘要/Abstract

摘要： 序列推荐是根据用户和项目的历史交互记录对用户兴趣建模，进行下一项目推荐。序列对用户兴趣建模通常分为长期兴趣依赖和短期兴趣依赖。现有的方法或按交互顺序先后将序列分割，分别对长短期兴趣依赖建模，割裂地对用户兴趣建模；或以不同的特征提取技术并行提取同一段交互序列的特征，获得全局和局部的兴趣表示，忽略不同时刻的用户意图存在该时刻行为上下文中这一事实。本文提出一种利用动态卷积和自注意力构建动态兴趣的序列推荐模型DConvSA。使用动态卷积提取局部动态兴趣，根据不同上下文项目生成卷积核，自适应计算项目的重要性。结合自注意力机制，获得全局显著项目依赖。以显式的方式融合每个时刻的全局和局部兴趣依赖，从而更好地对不同时刻兴趣间的联系建模。在3个公开数据集上进行实验，结果表明，其召回率、平均倒数排名和归一化折损累计增益在MovieLens-1M数据集上至少提升1.53%、3.77%和3.28%，在Amazon Beauty数据集上至少提升1.86%、1.94%和2.46%，在Steam数据集上至少提升0.22%、0.97%和1.08%。

Abstract: Sequence recommendation dynamically models user interests according to the historical interaction records of users and items， and recommends next item. The sequence modeling user interests is usually divided into long-term interest dependency and short-term interest dependency. The existing methods either divide the sequence according to the interaction order， respectively model the long-term and short-term interest dependence， separately model the user interest， or extract the features of the same interactive sequence in parallel with different feature extraction technologies to obtain the global and local interest representation， ignoring the fact that the user intention at different times exists in the behavior context at that time. This paper proposes DConvSA to model dynamic interest by using dynamic convolution and self-attention. Dynamic convolution is used to extract local dynamic interest， and convolution kernel is generated according to different context items to adaptively calculate the importance of items. Combined with the self-attention mechanism， the overall significant item dependency is obtained. At the same time， the global and local interest dependencies at each time are fused in an explicit way to better model the relationship between interests at different times. Experiments are conducted on three public datasets， using recall rate， average reciprocal ranking and normalized cumulative gain for performance evaluation. The results show that the recall rate， mean reciprocal ranking and normalized discounted cumulative gain increased by at least of 1.53%， 3.77% and 3.28% on the MovieLens-1M dataset， 1.86%， 1.94% and 2.46% on the Amazon Beauty dataset， and 0.22%， 0.97% and 1.08% on the Steam dataset.

Key words: Key words: sequence recommendation, dynamic convolution, self-attention, local interest, global interest

中图分类号:

TP391

郑海利, 陈平华. 基于动态卷积和自注意力的序列推荐模型[J]. 计算机与现代化, 2023, 0(10): 9-16.

ZHENG Hai-li, CHEN Ping-hua. Sequence Recommendation Model Based on Dynamic Convolution and Self-attention[J]. Computer and Modernization, 2023, 0(10): 9-16.

参考文献［28］

［1］	SARWAR B， KARYPIS G， KONSTAN J， et al. Item-based collaborative filtering recommendation algorithms［C］// Proceedings of the 10th International Conference on World Wide Web. 2001:285-295.
［2］	TANG J X， WANG K. Personalized top-N sequential recommendation via convolutional sequence embedding［C］// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 2018:565-573.
［3］	YUAN F J， KARATZOGLOU A， ARAPAKIS I， et al. A simple convolutional generative network for next item recommendation［C］// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019:582-590.
［4］	肖妍，霍林. 基于复合卷积和自注意力的会话推荐［J/OL］. 计算机工程与应用:1-13（2022-05-25）［2022-10-22］. http://kns.cnki.net/kcms/detail/11.2127.tp.20220523.
	1706.022.html.
［5］	LIU Q， ZENG Y F， MOKHOSI R， et al. STAMP: Short-term attention/memory priority model for session-based recommendation［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2018:1831-1839.
［6］	JIANG J Y， ZHANG P Y， LUO Y T， et al. AdaMCT: Adaptive mixture of CNN-Transformer for sequential recommendation［J］. arXiv preprint arXiv:2205.08776， 2022.
［7］	RENDLE S， FREUDENTHALER C， SCHMIDT-THIEME L. Factorizing personalized Markov chains for next-basket recommendation［C］// Proceedings of the 19th International Conference on World Wide Web. 2010:811-820.
［8］	HIDASI B， KARATZOGLOU A， BALTRUNAS L， et al. Session-based recommendations with recurrent neural networks［J］. arXiv preprint arXiv:1511.06939， 2015.
［9］	LI J， REN P J， CHEN Z M， et al. Neural attentive session-based recommendation［C］// Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 2017:1419-1428.
［10］	YUAN W H， WANG H， YU X M， et al. Attention-based context-aware sequential recommendation model［J］. Information Sciences， 2020，510:122-134.
［11］	王燕，范林，赵妮妮. 利用门控网络构建用户动态兴趣的序列推荐模型［J］. 计算机工程， 2022，48（8）:283-291.
［12］	YAN A， CHENG S， KANG W C， et al. CosRec: 2D convolutional neural networks for sequential recommendation［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019:2173-2176.
［13］	ZHANG Q， WU B， SUN Z C， et al. Gating augmented capsule network for sequential recommendation［J］. Knowledge-based Systems， 2022，247. DOI: 10.1016/j.knosys.2022.108817.
［14］	KANG W C， MCAULEY J. Self-attentive sequential recommendation［C］// Proceedings of the 2018 IEEE International Conference on Data Mining. 2018:197-206.
［15］	杜永萍，牛晋宇，王陆霖，等. 基于时间卷积注意力神经网络的序列推荐模型［J］. 模式识别与人工智能， 2022，35（5）:472-480.
［16］	SUN F， LIU J， WU J， et al. BERT4Rec: Sequential recommendation with bidirectional encoder representations from Transformer［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019:1441-1450.
［17］	WU L W， LI S Q， HSIEH C J， et al. SSE-PT: Sequential recommendation via personalized Transformer［C］// Proceedings of the 14th ACM Conference on Recommender Systems. 2020:328-337.
［18］	RASHED A， ELSAYED S， SCHMIDT-THIEME L. Context and attribute-aware sequential recommendation via cross-attention［C］// Proceedings of the 16th ACM Conference on Recommender Systems. 2022:71-80.
［19］	HU B B， SHI C， ZHAO W X， et al. Local and global information fusion for top-N recommendation in heterogeneous information network［C］// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018:1683-1686.
［20］	SONG J， WANG Y Y， TANG S L， et al. Local-global memory neural network for medication prediction［J］. IEEE Transactions on Neural Networks and Learning Systems， 2021，32（4）:1723-1736.
［21］	ZHANG S， TAY Y， YAO L N， et al. Next item recommendation with self-attentive metric learning［DB/OL］. ［2022-10-22］. https://recnlp2019.github.io/papers/RecNLP2019_paper_21.pdf.
［22］	ZHANG L X， WANG P S， LI J C， et al. Attentive hybrid recurrent neural networks for sequential recommendation［J］. Neural Computing and Applications， 2021，33（17）:11091-11105.
［23］	WANG F， FENG W S. Multi-interest sequence recommendation algorithm based on BERT［C］// Proceedings of the 4th International Conference on Computer Science and Software Engineering. 2021:13-17.
［24］	DU Y P， PENG Z， NIU J Y， et al. A unified hierarchical attention framework for sequential recommendation by fusing long and short-term preferences［J］. Expert Systems with Applications， 2022，201. DOI: 10.1016/j.eswa.2022.117102.
［25］	LIN J， PAN W K， MING Z. FISSA: Fusing item similarity models with self-attention networks for sequential recommendation［C］// Proceedings of the 14th ACM Conference on Recommender Systems. 2020:130-139.
［26］	JIANG Z H， YU W H， ZHOU D Q， et al. ConvBERT: Improving BERT with span-based dynamic convolution［C］// Proceedings of the 2020 IEEE International Conference on Neural Information Processing Systems. 2020:12837-12848.
［27］	RENDLE S， FREUDENTHALER C， GANTNER Z， et al. BPR: Bayesian personalized ranking from implicit feedback［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009:452-461.
［28］	HE X N， LIAO L Z， ZHANG H W， et al. Neural collaborative filtering［C］// Proceedings of the 26th International Conference on World Wide Web. 2017:173-182.

[1]	李亚平, 王军防, 余红梅, 窦一民, 肖媛, 田继林. Regformer：基于稀疏注意力的输油管道水力压降预测方法[J]. 计算机与现代化, 2024, 0(01): 59-66.
[2]	龚轩, 郭中华, 陈旺. 基于CA-TransUNet的遥感图像道路分割[J]. 计算机与现代化, 2023, 0(07): 112-118.
[3]	夏义春, 李汪根, 李豆豆, 葛英奎, 王志格. 结合注意力机制和图神经网络的CTR预估模型[J]. 计算机与现代化, 2023, 0(03): 29-37.
[4]	乔善宝, 高永彬, 黄勃, 余文俊. 基于条件卷积与极化自注意力的单目深度与位姿估计[J]. 计算机与现代化, 2023, 0(02): 34-39.
[5]	栗伟松, 汤敏芳, 何征岭, 王鹏, 杜利东, 方震, 陈贤祥. 基于自注意力机制和单导联心电信号的自动睡眠分期算法#br#[J]. 计算机与现代化, 2022, 0(12): 50-59.
[6]	何晨曦, 王正勇, 卿粼波, 何小海, 吴小强. 基于全局自注意力的小麦图像识别[J]. 计算机与现代化, 2022, 0(04): 38-44.
[7]	万发洋, 于旭, 徐其江. 基于多头自注意力机制的深度缺陷分派模型[J]. 计算机与现代化, 2021, 0(11): 39-43.
[8]	宋爽, 陆鑫达. 基于BERT与图像自注意力机制的文本匹配模型[J]. 计算机与现代化, 2021, 0(11): 12-16.
[9]	潘陈听, 谭晓阳, . 复杂背景下基于深度学习的视频动作识别[J]. 计算机与现代化, 2020, 0(07): 97-103.
[10]	徐龙. 基于自注意力和胶囊网络的短文本情感分析[J]. 计算机与现代化, 2020, 0(07): 61-64.