Computer and Modernization ›› 2023, Vol. 0 ›› Issue (03): 29-37.

Previous Articles     Next Articles

CTR Prediction Model Combining Attention Mechanism and Graph Neural Network

  

  1. (School of Computer and Information, Anhui Normal University, Wuhu 241002, China)
  • Online:2023-04-17 Published:2023-04-17

Abstract: Most CTR prediction algorithms initialize the feature embedding as a fixed dimension, ignoring the low popularity of the long tail feature. Setting it to the same length as the head object embedding vector will lead to unbalanced model training and affect the final recommendation results. Based on this, this paper first uses an end-to-end differentiable framework, which can automatically select different embedded dimensions according to the popularity of features. Secondly, this paper introduces squeeze excitation network mechanism and multi-head self-attention mechanism with residual connection to dynamically learn the importance of features and identify important feature combinations from different angles, and then uses graph neural network to explicitly model the second-order feature interaction instead of traditional inner product and Hadamard product. Finally, in order to further improve the performance, this paper combines the DNN component with the shallow model to form the depth model, uses the Bayesian optimization algorithm to select a set of super parameters for the depth model to avoid the complex parameter adjustment process, and the experimental results on two benchmark datasets verify the effectiveness of the model.

Key words: CTR prediction, automatic embedded search, squeeze excitation network, multi-head self-attention mechanism, graph neural network, Bayesian optimization