Intent-based Lightweight Self-Attention Network for Sequential Recommendation

doi:10.3969/j.issn.1006-2475.2024.12.001

Abstract

Abstract: The parameters of the self-attention calculation mechanism in the existing sequence recommendation models are too large， and there is insufficient preference information in the user's shopping intention. This paper proposes an intent-based lightweight self-attention network for sequential recommendation. On the basis of the traditional product sequence embedding， the model introduces intention sequence embedding to further explore the conversion patterns between sequences. At the same time， in order to reduce the computational complexity of self-attention between pairwise products/intentions in the sequence， a convolutional segmentation sampling module is designed to divide the user behavior sequence and intention sequence into multiple segments， mapping user interests to multiple sequence segments. Comparative experiments are conducted on three public datasets， MovieLens-1M， Yelp， and Amazon-Books. Compared with baseline models， the self-attention mechanism is applied to capture the dependency between segments， effectively reducing the number of parameters. The results show that the hit rate， normalized discounted cumulative gain and mean reciprocal ranking are increased by 5.32%， 4.40% and 5.51% on the MovieLens-1M dataset， 30.93%， 22.73% and 28.84% on the Yelp dataset， and 7.78%， 11.55% and 13.98% on the Amazon-Books dataset， which verify the effectiveness of the model proposed in this paper.

Key words: sequence recommendation； intent recommendation； convolution neural network； self-attention mechanisms ,

CLC Number:

TP391

HE Sida, CHEN Pinghua. Intent-based Lightweight Self-Attention Network for Sequential Recommendation[J]. Computer and Modernization, 2024, 0(12): 1-9.

References

［1］汪菁瑶，吴国栋，范维成，等. 用户行为序列个性化推荐研究综述［J］. 小型微型计算机系统， 2022，43（5）：921-935.
［2］ WANG S J， ZHANG Q， HU L， et al. Sequential/session-based recommendations： Challenges， approaches， applications and opportunities［C］// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM， 2022：3425-3428.
［3］ WANG S J， HU L， WANG Y， et al. Sequential recommender systems： Challenges， progress and prospects［J］. arXiv preprint arXiv：2001.04830， 2019.
［4］ CEN Y K， ZHANG J W， ZOU X， et al. Controllable multi-interest framework for recommendation［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM， 2020：2942-2951.
［5］ RENDLE S， FREUDENTHALER C， SCHMIDT-THIEME L. Factorizing personalized Markov chains for next-basket recommendation［C］// Proceedings of the 19th International Conference on World Wide Web. ACM， 2010：811-820.
［6］ WU C Y， AHMED A， BEUTEL A， et al. Recurrent recommender networks［C］// Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM， 2017：495-503.
［7］ CHO K， VAN MERRIËNBOER B， GULCEHRE C， et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation［J］. arXiv preprint arXiv：1406.1078， 2014.
［8］ HIDASI B， KARATZOGLOU A， BALTRUNAS L， et al. Session-based recommendations with recurrent neural networks［J］. arXiv preprint arXiv：1511.06939， 2015.
［9］ SONG Y， LEE J G. Augmenting recurrent neural networks with high-order user-contextual preference for session-based recommendation［J］. arXiv preprint arXiv：1805.02983，
2018.
［10］ KUMAR V， KHATTAR D， GUPTA S， et al. Deep neural architecture for news recommendation［C］// The 8th International Conference of the CLEF Initiative， CEUR Workshop Proceedings. 2017.
［11］ DAI H J， WANG Y C， TRIVEDI R， et al. Deep coevolutionary network： Embedding user and item features for recommendation［J］. arXiv preprint arXiv：1609.03675， 2016.
［12］ VASSØY B， RUOCCO M， DE SOUZA DA SILVA E， et al. Time is of the essence： A joint hierarchical RNN and point process model for time and item predictions［C］// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. ACM，2019：591-599.
［13］ TANG J X， WANG K. Personalized top-n sequential recommendation via convolutional sequence embedding［C］// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM， 2018：565-573.
［14］ HSU K C， CHOU S Y， YANG Y H， et al. Neural network based next-song recommendation［J］. arXiv preprint arXiv：1606.07722， 2016.
［15］ TUAN T X， PHUONG T M. 3D convolutional networks for session-based recommendation with content features［C］//Proceedings of the 11th ACM Conference on Recommender Systems. ACM， 2017：138-146.
［16］ YAN A， CHENG S， KANG W C， et al. CosRec：2D convolutional neural networks for sequential recommendation［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM， 2019：2173-2176.
［17］ YUAN F J， HE X N， JIANG H C， et al. Future data helps training： Modeling future contexts for session-based recommendation［C］// Proceedings of The Web Conference 2020. ACM， 2020：303-313.
［18］ YU F， KOLTUN V. Multi-scale context aggregation by dilated convolutions［J］. arXiv preprint arXiv：1511.07122， 2015.
［19］ YAKHCHI S， BEHEHSTI A， GHAFARI S M， et al. A convolutional attention network for unifying general and sequential recommenders［J］. Information Processing & Management， 2022， 59（1）. DOI: 10.1016/j.ipm.2020.102755.
［20］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc， 2017：6000-6010.
［21］ KANG W C， MCAULEY J. Self-attentive sequential recommendation［C］//2018 IEEE International Conference on Data Mining （ICDM）. IEEE， 2018：197-206.
［22］ ZHANG S， TAY Y， YAO L N， et al. Next item recommendation with self-attention［J］. arXiv preprint arXiv：1808.
06414， 2018.
［23］ LI J C， WANG Y J， MCAULEY J. Time interval aware self-attention for sequential recommendation［C］// Proceedings of the 13th International Conference on Web Search and Data Mining. ACM， 2020：322-330.
［24］ ALAPARTHI S， MISHRA M. Bidirectional encoder representations from transformers （BERT）： A sentiment analysis odyssey［J］. arXiv preprint arXiv：2007.01127， 2020.
［25］ SUN F， LIU J， WU J， et al. BERT4Rec： Sequential recommendation with bidirectional encoder representations from transformer［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM， 2019：1441-1450.
［26］ WU L W， LI S Q， HSIEH C J， et al. SSE-PT： Sequential recommendation via personalized transformer［C］// Proceedings of the 14th ACM Conference on Recommender Systems. ACM， 2020：328-337.
［27］ FAN X Y， LIU Z， LIAN J X， et al. Lighter and better： Low-rank decomposed self-attention networks for next-item recommendation［C］// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021：1733-1737.
［28］ CHEN W Y， REN P J， CAI F， et al. Improving end-to-end sequential recommendations with intent-aware diversification［C］// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. ACM， 2020：175-184.
［29］ TANJIM M， SU C Z， BENJAMIN E， et al. Attentive sequential models of latent intent for next item recommendation［C］// Proceedings of The Web Conference2020. ACM， 2020：2528-2534.
［30］ LI H Y， WANG X， ZHANG Z W， et al. Intention-aware sequential recommendation with structured intent transition［J］. IEEE Transactions on Knowledge and Data Engineering， 2022，34（11）：5403-5414.
［31］ ZHAO W X， HOU Y P， PAN X Y， et al. RecBole 2.0： Towards a more up-to-date recommendation library［C］// Proceedings of the 31st ACM International Conference on Information & Knowledge Management. ACM， 2022： 4722-4726.

[1]	ZHENG Jiuchao, ZHAO Xinyuan. Entity Linking Method Based on Topics and Description Information [J]. Computer and Modernization, 2024, 0(12): 10-14.
[2]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[3]	WANG Xiaohang1, LI Yongjie1, YU Lei1, FAN Xiao2. A Method of Using Compound Event Probability Operation to Solve Problem of Negative Information Blocking Maximization [J]. Computer and Modernization, 2024, 0(12): 24-33.
[4]	HUANG Tingpei1, MA Lubiao1, LI Shibao2, LIU Jianhang1. Gesture Recognition Method Based on WiFi and Prototypical Network [J]. Computer and Modernization, 2024, 0(12): 34-39.
[5]	LYU Meijing1, NIAN Mei1, ZHANG Jun1, 2, FU Lusen1. Anomaly Detection of Network Traffic Based on Autoencoder [J]. Computer and Modernization, 2024, 0(12): 40-44.
[6]	ZHANG Simin, LIU Xinmei, YIN Junling, LI Baoling. PCB Defect Detection Method Based on Improved YOLOv7 [J]. Computer and Modernization, 2024, 0(12): 45-52.
[7]	ZHANG Xiaodong1, BAI Guangzhi1, LI Min1, LI Haoyang2. Oil and Gas Well Production Prediction Model Based on Empirical Wavelet Transform [J]. Computer and Modernization, 2024, 0(12): 53-58.
[8]	WANG Haiyang, GONG Tongxin, YANG Jintao, CHEN Zailong. Short-term Load Forecasting in Industrial Parks with Multi-scale Time Coding [J]. Computer and Modernization, 2024, 0(12): 59-65.
[9]	LIU Yunhai1, Feng Guang1, WU Xiaoting2, YANG Qun2 . Safety Helmet Wearing Detection Algorithm for Complex Construction Scenes [J]. Computer and Modernization, 2024, 0(12): 66-71.
[10]	LIU Baobao, YANG Jingjing, TAO Lu, WANG Heying . DSMSC Based on Attention Mechanism for Remote Sensing Image Scene Classification [J]. Computer and Modernization, 2024, 0(12): 72-77.
[11]	GU Yue, DENG Songfeng, SHEN Ji, MU Wentao, ZHAO Enqi. SAR Ship Detection Algorithm Based on Improved YOLOv8 [J]. Computer and Modernization, 2024, 0(12): 78-83.
[12]	WANG Yanyuan, MAO Zhengchong. Detection and Recognition Algorithms for Chinese and English Scene Text Images [J]. Computer and Modernization, 2024, 0(12): 84-90.
[13]	WAN Bing1, 2, 3, ZHAO Wentao4, PAN Duotao1, ZHAO Zhengtao2, 3, SUN Zhaoyang2, 3, YU Jiancheng2, 3 . Design of Semi-physical Simulation Test System for Unmanned Sailboat [J]. Computer and Modernization, 2024, 0(12): 91-99.
[14]	CHEN Liang, LI Cheng, YI Wei, XIONG Wei, WANG Xiaofan, TANG Haidong. Helmet Wearing Detection in Electric Power Field Based on#br# Millimeter-wave Radar and Visual Fusion [J]. Computer and Modernization, 2024, 0(12): 100-107.
[15]	WU Xiuling1, ZHOU Sheng1, WANG Chunjuan1, YU Cuizhuo2, LIU Hao3. Research Progress in Ultra Short-term Power Load Forecasting Technology [J]. Computer and Modernization, 2024, 0(12): 108-115.