基于用户动态兴趣的视频点击率预测模型

摘要/Abstract

摘要： 针对经典的点击预测模型无法捕捉用户动态兴趣和分析特征低阶高阶交互困难的问题，提出一种基于用户动态兴趣的视频点击预测模型。该模型首先将离散的数据经嵌入过程后映射成易于操作的低维连续向量；为捕捉用户动态兴趣变化，引入transformer模型，同时分析用户点击视频序列与待预测的候选视频，抽取行为序列中的视频与待推荐视频之间的相互作用；为深入挖掘用户点击行为背后的隐式特征交互，引入DeepFM网络并在网络结构上进行优化改进，使模型更加适合顺序依赖的点击数据。实验结果表明本文提出改进的模型预测精度优于在点击率预测方面比较具有代表性的深度分解机等模型，同时引入transformer机制可以明显提升点击率预测的精度。

关键词: 点击率预测, 用户兴趣, 特征交互, 因式分解机, transformer

Abstract: Aiming at the problem that the classic click prediction model cannot capture the users dynamic interest and analyze the characteristics of low-level and high-level interaction, this paper proposes a video click prediction model based on the users dynamic interest. The model first maps the discrete data into low-dimensional continuous vectors that are easy to operate after embedding. In order to capture the users dynamic interest changes, the transformer model is introduced to analyze the video sequence clicked by users and the candidate video to be predicted, and the interaction between the video in the behavior sequence and the video to be recommended is extracted. In order to dig deeper into the implicit feature interaction behind the users click behavior, the DeepFM network is introduced and the network structure is optimized and improved to make the model more suitable for sequence-dependent click data. The experimental results show that the prediction accuracy extension of the model proposed and improved in this paper is better than that of the typical deep decomposition model in click rate prediction, and the release of the transformer mechanism can significantly improve the accuracy of click rate prediction.

Key words: click-through rate prediction, user interest, feature interaction, factorization machine, transformer

杨佳雪, 彭国争, 韩立新. 基于用户动态兴趣的视频点击率预测模型[J]. 计算机与现代化, 2021, 0(03): 82-87.

YANG Jia-xue, PENG Guo-zheng, HAN Li-xin. Video Click-through Rate Prediction Model Based on Users Dynamic Interests[J]. Computer and Modernization, 2021, 0(03): 82-87.

参考文献［23］

［1］	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need［C］// Advances in Neural Information Processing Systems. 2017:5998-6008.
［2］	GUO H F, TANG R M, YE Y M, et al. DeepFM: A factorization-machine based neural network for CTR prediction［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017:1725-1731.
［3］	RENDLE S. Factorization machines［C］// 2010 IEEE International Conference on Data Mining. 2010:995-1000.
［4］	JUAN Y, ZHUANG Y, CHIN W S, et al. Field-aware factorization machines for CTR prediction［C］// Proceedings of the 10th ACM Conference on Recommender Systems. 2016:43-50.
［5］	WANG P, PUTERMAN M L. Mixed logistic regression models［J］. Journal of Agricultural, Biological, and Environmental Statistics, 1998,3（2）:175-200.
［6］	ZHANG W N, DU T M, WANG J. Deep learning over multi-field categorical data［C］// European Conference on Information Retrieval. 2016:45-57.
［7］	HE X N, CHUA T S. Neural factorization machines for sparse predictive analytics［C］// Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017:355-364.
［8］	XIAO J, YE H, HE X N, et al. Attentional factorization machines: Learning the weight of feature interactions via attention networks［J］. Machine Learning, 2017，arXiv:1708.04617.
［9］	CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems［C］// Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 2016:7-10.
［10］	QU Y R, CAI H, REN K, et al. Product-based neural networks for user response prediction［C］// 2016 IEEE 16th International Conference on Data Mining. 2016:1149-1154.
［11］	WANG R X, FU B, FU G, et al. Deep & cross network for ad click predictions［C］// Proceedings of the ADKDD17. 2017:1-7.
［12］	LIU Q, YU F, WU S, et al. A convolutional click prediction model［C］// Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015:1743-1746.
［13］	ZHOU G R, ZHU X Q, SONG C R, et al. Deep interest network for click-through rate prediction［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018:1059-1068.
［14］	CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling［J］. Neural and Evolutionary Computing, 2014,arXiv:1412.3555.
［15］	ZHOU G R, MOU N, FAN Y, et al. Deep interest evolution network for click-through rate prediction［C］// Proceedings of the AAAI Conference on Artificial Intelligence. 2019,33:5941-5948.
［16］	SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014:3104-3112.
［17］	BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate［J］. Computation and Language, 2014,arXiv:1409.0473.
［18］	CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation［J］. Computation and Language, 2014,arXiv:1406.1078.
［19］	HOCHREITER S, SCHMIDHUBERJ. Long short-term memory［J］. Neural Computation, 1997,9（8）:1735-1780.
［20］	SONG W P, SHI C, XIAO Z P, et al. Autoint: Automatic feature interaction learning via self-attentive neural networks［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019:1161-1170.
［21］	CHEN Q W, ZHAO H, LI W, et al. Behavior sequence transformer for e-commerce recommendation in alibaba［C］// Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. 2019:1-4.
［22］	PEI C H, ZHANG Y, ZHANG Y F, et al. Personalized re-ranking for recommendation［C］// Proceedings of the 13th ACM Conference on Recommender Systems. 2019:3-11.
［23］	LIAN J X, ZHOU X H, ZHANG F Z, et al. XDeepFM: Combining explicit and implicit feature interactions for recommender systems［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018:1754-1763.

[1]	付鸿林, 张太红, 杨雅婷, 艾孜麦提·艾瓦尼尔, 马博. 基于生成对抗网络的维语场景文字修改网络[J]. 计算机与现代化, 2024, 0(01): 41-46.
[2]	李亚平, 王军防, 余红梅, 窦一民, 肖媛, 田继林. Regformer：基于稀疏注意力的输油管道水力压降预测方法[J]. 计算机与现代化, 2024, 0(01): 59-66.
[3]	杨柳青, 王冲. 基于极大熵的Web服务资源个性化推荐方法[J]. 计算机与现代化, 2023, 0(09): 32-37.
[4]	许叶彤, 耿信哲, 赵伟强, 张月, 宁海龙, 雷涛. 基于CNN-Transformer混合结构的遥感影像变化检测模型[J]. 计算机与现代化, 2023, 0(07): 79-85.
[5]	刘静, 陈金广. 基于通道注意力和Transformer的图像标题生成方法[J]. 计算机与现代化, 2023, 0(05): 8-12.
[6]	夏义春, 李汪根, 李豆豆, 葛英奎, 王志格. 结合注意力机制和图神经网络的CTR预估模型[J]. 计算机与现代化, 2023, 0(03): 29-37.
[7]	王洪义, 孔梅梅, 徐荣青. 基于改进YOLOV5的火焰检测算法[J]. 计算机与现代化, 2023, 0(01): 103-107.
[8]	林椹尠, 屈嘉欣, 罗亮. 基于改进的Transformer_decoder的增强图像描述[J]. 计算机与现代化, 2023, 0(01): 7-12.
[9]	李鑫, 任德均, 任秋霖, 曹林杰, 闫宗一. 基于Retinanet的轮毂焊缝检测定位方法[J]. 计算机与现代化, 2022, 0(09): 60-67.
[10]	陈彤, 周登文. 基于多级Transformer的超大倍率重建网络:参考图像超分辨率[J]. 计算机与现代化, 2022, 0(08): 121-126.
[11]	何晨曦, 王正勇, 卿粼波, 何小海, 吴小强. 基于全局自注意力的小麦图像识别[J]. 计算机与现代化, 2022, 0(04): 38-44.
[12]	冯茹嘉, 张海军, 潘伟民. 基于情感分析和Transformer模型的微博谣言检测[J]. 计算机与现代化, 2021, 0(10): 1-7.
[13]	李欣晔, 张承强, 周雄图, 郭太良, 张永爱. 多场景融合的细粒度图像描述生成算法[J]. 计算机与现代化, 2021, 0(09): 1-6.
[14]	申强华. 基于多变量Probit回归的用户兴趣挖掘方法[J]. 计算机与现代化, 2017, 0(6): 25-29.
[15]	谢梓琪，谢颖华. 一种基于数据挖掘的SNS用户兴趣模型[J]. 计算机与现代化, 2014, 0(2): 24-27+31.