Abstract: In order to solve the problems of target occlusion and complex background in target tracking, a target tracking algorithm based on multimodal data is proposed. First, pixel-level fusion of each modal data is performed to reduce the impact of insufficient information in single-modal data on the tracking results. Then, different features are extracted and filtered from the fused image. At the same time, in order to solve the problem of model tracking failure caused by a single model drift, the response graph obtained by filtering is merged at the decision level. Finally, the tracking result is obtained according to the peak value of the fused response graph. In addition, an occlusion detection module is added in the tracking process to enhance the model robustness. The evaluation of the algorithm on the Princeton tracking benchmark shows that, compared with other mainstream algorithms, the target tracking algorithm based on multimodal data improves the tracking accuracy on target occlusion videos by 8.4% and the coincidence success rate by 3.3%. It has a good anti-occlusion effect.

Key words: computer vision, object tracking, correlation filter, multimodal fusion, occlusion detecting