Computer and Modernization ›› 2023, Vol. 0 ›› Issue (08): 44-53.doi: 10.3969/j.issn.1006-2475.2023.08.008

Previous Articles     Next Articles

Cross Modal Hash Retrieval Based on Attention Mechanism and Semantic Similarity

  

  1. (School of Computer Science, South China Normal University, Guangzhou 510631, China)
  • Online:2023-08-30 Published:2023-09-13

Abstract: Abstract: Nowadays, cross-modal hash retrieval has been widely and successfully used in multimedia similarity search applications. There are two challenged questions in deep hash retrieval methods:1)How to measure multiple modal’s similarity more accurately. 2)How to fuse multiple modal’s features to gain more abundant feature representations, so as to avoid key information loss. Therefore, in order to solve these two problems, we propose a novel cross-modal hashing method, called cross-modal hash retrieval model based on attention mechanism and semantic similarity (ASSH), by defining a new multi-label similarity measurement method to distinguish the importance of different labels, designing an attention fusion module to fuse the features and enhance the interaction between different modal. Experimental results demonstrate that the proposed method outperforms the previous methods in all problem modes on the three common datasets MIRFLICKR-25K, NUS-WIDE and IAPR TC-12. Compared to the state-of-the-art method, when the hash code length is 16 bit, the mean Average Precision (mAP) is improved by 1.1% and 0.63%. At the same time, the ablation experiment also fully proved the effectiveness of the method.

Key words: Key words: cross modal retrieval, attention mechanism, similarity matrix, hash retrieval, feature fusion

CLC Number: