基于改进YOLOX和新型数据关联方式的无人机#br#
多目标跟踪方法

doi:10.3969/j.issn.1006-2475.2024.08.011

摘要/Abstract

摘要： 无人机视频中的多目标跟踪是一项重要的计算机视觉任务，在各个领域有着广泛的应用。针对无人机视频场景中目标遮挡、小目标、背景复杂多变等挑战，本文提出一种改进的无人机多目标跟踪模型。首先，本文对YOLOX进行改进，将Swin Transformer集成到网络中以增强全局信息提取能力，并增加一个额外的检测头来改善对小目标的检测能力，此外引入CBAM注意力模块来增强聚焦有用信息的能力。数据关联阶段，本文采用一种新型数据关联方式，保留所有检测框，并根据置信度将其划分为高分检测框和低分检测框，对高分检测框与跟踪轨迹进行第一次关联，将未匹配轨迹与低分检测框进行二次关联。在公开数据集VisDrone2021和UAVDT上的实验结果表明，本文方法在无人机多目标跟踪场景中具有较高的优越性和鲁棒性。

关键词: 多目标跟踪, 无人机视频, 注意力机制, 数据关联

Abstract: Multi-object tracking in UAV videos is a crucial computer vision task with extensive applications across various domains. To address the challenges of occlusions， small objects， and complex， varying backgrounds in UAV video scenes， an improved UAV multi-object tracking model is proposed. This paper improves the YOLOX network by integrating the Swin Transformer to enhance global information extraction capabilities and adding an additional detection head to boost the detection performance of small objects. Furthermore， this paper introduces the CBAM attention module to focus on informative features. In the data association stage， this paper adopts a new data association approach that retains all detection boxes， categorizing them into high-scoring and low-scoring detection boxes based on their confidence scores. The first association is performed between high-scoring detection boxes and tracking trajectories， while the second association is performed between unmatched trajectories and low-scoring detection boxes. Experimental results on the public datasets VisDrone2021 and UAVDT demonstrate that the proposed method exhibits relatively high superiority and robustness in UAV multi-object tracking scenarios.

Key words: multi-object tracking, unmanned aerial vehicles（UAV） videos, attention mechanism, data association

中图分类号:

TP391

付书岗1, 2, 3. 基于改进YOLOX和新型数据关联方式的无人机#br# 多目标跟踪方法[J]. 计算机与现代化, 2024, 0(08): 59-66.

FU Shugang1, 2, 3. Multi-object Tracking of UAV Based on Improved YOLOX and New Data Association Method[J]. Computer and Modernization, 2024, 0(08): 59-66.

参考文献

［1］ PATRICIO D I， RIEDER R. Computer vision and artificial intelligence in precision agriculture for grain crops： A systematic review［J］. Computers and Electronics in Agriculture， 2018，153：69-81.
［2］ VOULODIMOS A， DOULAMIS N， DOULAMIS A， et al. Deep learning for computer vision： A brief review［J］. Computational Intelligence and Neuroscience， 2018，2018. DOI： 10.1155/2018/7068349.
［3］ JANAI J， GUNEY F， BEHL A， et al. Computer vision for autonomous vehicles： Problems， datasets and state of the art［J］. Foundations and Trends in Computer Graphics and Vision， 2020，12（1-3）：1-308.
［4］ FERNANDEZ-SANJURJO M， MUCIENTES M， BREA V M. Real-time multiple object visual tracking for embedded GPU systems［J］. IEEE Internet of Things Journal， 2021，8（11）：9177-9188.
［5］ CHEN T， PENNISI A， LI Z， et al. A hierarchical association framework for multi-object tracking in airborne videos［J］. Remote Sensing， 2018，10（9）. DOI： 10.3390/rs10091347.
［6］ KOK K Y， RAJENDRAN P. Differential-evolution control parameter optimization for unmanned aerial vehicle path planning［J］. PLoS One， 2016，11（3）. DOI： 10.1371/journal.pone.0150558.
［7］ DELIBASOGLU I. UAV images dataset for moving object detection from moving cameras［J］. arXiv preprint arXiv：2103.11460， 2021.
［8］ ZHU P F， WEN L Y， DU D W， et al. Detection and tracking meet drones challenge［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2022，44（11）：7380-7399.
［9］ AZIMI S M， KRAUS M， BAHMANYAR R， et al. Multiple pedestrians and vehicles tracking in aerial imagery： A comprehensive study［J］. arXiv preprint arXiv：2010.09689， 2020.
［10］ WEN L Y， DU D W， ZHU P F， et al. Detection， tracking， and counting meets drones in crowds： A benchmark［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2021：7808-7817.
［11］ VARGA L A， KIEFER B， MESSMER M， et al. SeaDronesSee： A maritime benchmark for detecting humans in open water［C］// Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE， 2022：3686-3696.
［12］ KAMPFFMEYER M， SALBERG A B， JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE， 2016：680-688.
［13］ AHMED I， AHMAD M， AHMAD A， et al. Top view multiple people tracking by detection using deep SORT and YOLOv3 with transfer learning： Within 5G infrastructure［J］. International Journal of Machine Learning and Cybernetics， 2021，12（11）：3053-3067.
［14］ WANG Z Y， MIAO D Q， ZHAO C R， et al. A robust long-term pedestrian tracking-by-detection algorithm based on three-way decision［C］// Proceedings of the 2019 International Joint Conference on Rough Sets. Springer， 2019：522-533.
［15］ XIE Y F， HUANG Y， SONG T L. Iterative joint integrated probabilistic data association filter for multiple-detection multiple-target tracking［J］. Digital Signal Processing， 2018，72：232-243.
［16］ WANG Z D， ZHENG L， LIU Y X， et al. Towards real-time multi-object tracking［C］// Proceedings of the 16th European Conference on Computer Vision. Springer， 2020：107-122.
［17］ MUNJAL B， AFTAB A R， AMIN S， et al. Joint detection and tracking in videos with identification features［J］. Image and Vision Computing， 2020，100. DOI： 10.1016/j.imavis.2020.103932.
［18］ LIN X W， GUO Y A， WANG J Q. Global correlation network： End-to-end joint multi-object detection and tracking［J］. arXiv preprint arXiv：2103.12511， 2021.
［19］ SUN Z H， CHEN J， CHAO L， et al. A survey of multiple pedestrian tracking based on tracking-by-detection framework［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2021，31（5）：1819-1833.
［20］ WU J L， CAO J L， SONG L C， et al. Track to detect and segment： An online multi-object tracker［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2021：12347-12356.
［21］ WOJKE N， BEWLEY A， PAULUS D. Simple online and realtime tracking with a deep association metric［C］// Proceedings of the 2017 IEEE International Conference on Image Processing （ICIP）. IEEE， 2017：3645-3649.
［22］ GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2014：580-587.
［23］ REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： Towards real-time object detection with region proposal networks［J］. arXiv preprint arXiv：1506.01497， 2015.
［24］ LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. IEEE， 2017：2999-3007.
［25］ ZHOU X Y， KOLTUN V， KRAHENBUHL P. Tracking objects as points［C］// Proceedings of the 16th European Conference on Computer Vision. Springer， 2020：474-490.
［26］ REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： Unified， real-time object detection［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2016：779-788.
［27］ GE Z， LIU S T， WANG F， et al. YOLOX： Exceeding YOLO series in 2021［J］. arXiv preprint arXiv：2107.08430， 2021.
［28］ BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： Optimal speed and accuracy of object detection［J］. arXiv preprint arXiv：2004.10934， 2020.
［29］ JOCHER G， STOKEN A， BOROVEC J， et al. Ultralytics/YOLOv5： v5.0-YOLOv5-P6 1280 models， AWS， Supervise.ly and YouTube integrations［DB/OL］. （2021-04-11）［2023-03-19］. https：//ui.adsabs.harvard.edu/abs/2021zndo...4679653J/abstract.
［30］ LU Z C， RATHOD V， VOTEL R， et al. RetinaTrack： Online single stage joint detection and tracking［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2020：14656-14666.
［31］ ZHANG Y F， WANG C Y， WANG X G， et al. FairMOT： On the fairness of detection and re-identification in multiple object tracking［J］. International Journal of Computer Vision， 2021，129（11）：3069-3087.
［32］ CHU P， WANG J， YOU Q Z， et al. TransMOT： Spatial-temporal graph transformer for multiple object tracking［C］// Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE， 2023：4859-4869.
［33］ KALMAN R E. A new approach to linear filtering and prediction problems［J］. Journal of Basic Engineering， 1960，82（1）：35-45.
［34］ BEWLEY A， GE Z Y， OTT L， et al. Simple online and realtime tracking［C］// Proceedings of the 2016 IEEE International Conference on Image Processing （ICIP）. IEEE， 2016：3464-3468.
［35］ DU Y H， WAN J F， ZHAO Y Y， et al. GIAOTracker： A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. IEEE， 2021：2809-2819.
［36］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［J］. arXiv preprint arXiv：1706.03762， 2017.
［37］ WOO S， PARK J， LEE J Y， et al. CBAM： Convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision. Springer， 2018：3-19.
［38］ WANG C Y， LIAO H Y M， WU Y H， et al. CSPNet： A new backbone that can enhance learning capability of CNN［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE， 2020：1571-1580.
［39］ HE K M， ZHANG X Y， REN S Q， et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015，37（9）：1904-1916.
［40］ LIU S， QI L， QIN H F， et al. Path aggregation network for instance segmentation［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2018：8759-8768.
［41］ LIN T Y， DOLLAR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2017：936-944.
［42］ DOSOVITSKIY A， BEYER L， KOLESNIKOV A， et al. An image is worth 16x16 words： Transformers for image recognition at scale［J］. arXiv preprint arXiv：2010.11929， 2020.
［43］ CARION N， MASSA F， SYNNAEVE G， et al. End-to-end object detection with transformers［C］// Proceedings of the 16th European Conference on Computer Vision. Springer， 2020：213-229.
［44］ LIU Z， LIN Y T， CAO Y， et al. Swin transformer： Hierarchical vision transformer using shifted windows［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. IEEE， 2021：9992-10002.
［45］ ZHANG Y F， SUN P Z， JIANG Y， et al. ByteTrack： Multi-object tracking by associating every detection box［C］// Proceedings of the 17th European Conference on Computer Vision. Springer， 2022：1-21.
［46］ CHEN G L， WANG W G， HE Z J， et al. VisDrone-MOT2021： The vision meets drone multiple object tracking challenge results［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops. IEEE， 2021：2839-2846.
［47］ DU D W， QI Y K， YU H Y， et al. The unmanned aerial vehicle benchmark： Object detection and tracking［C］// Proceedings of the 2018 European Conference on Computer Vision. Springer， 2018：375-391.
［48］ CHEN L， AI H Z， ZHUANG Z J， et al. Real-time multiple people tracking with deeply learned candidate selection and person re-identification［C］// Proceedings of the 2018 IEEE International Conference on Multimedia and Expo （ICME）. IEEE， 2018. DOI： 10.1109/ICME.2018.8486597.
［49］ BOCHINSKI E， EISELEIN V， SIKORA T. High-speed tracking-by-detection without using image information［C］// Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance （AVSS）. IEEE， 2017. DOI： 10.1109/AVSS.2017.8078516.
［50］PIRSIAVASH H， RAMANAN D， FOWLKES C C. Globally-optimal greedy algorithms for tracking a variable number of objects［C］// Proceedings of the 2011 Conference on Computer Vision and Pattern Recognition （CVPR）. IEEE， 2011：1201-1208.
［51］ ZENG F G， DONG B， ZHANG Y A， et al. MOTR： End-to-end multiple-object tracking with transformer［C］// Proceedings of the 17th European Conference on Computer Vision. Springer， 2022：659-675.
［52］ MEINHARDT T， KIRILLOV A， LEAL-TAIXE L， et al. Trackformer： Multi-object tracking with transformers［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE， 2022：8834-8844.
［53］ MILAN A， ROTH S， SCHINDLER K. Continuous energy minimization for multitarget tracking［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2014，36（1）：58-72.
［54］ BAE S H， YOON K J. Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE， 2014：1218-1225.
［55］ SUN S J， AKHTAR N， SONG H S， et al. Deep affinity network for multiple object tracking［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021，43（1）：104-119.
［56］ XIANG Y， ALAHI A， SAVARESE S. Learning to track： Online multi-object tracking by decision making［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. IEEE， 2015：4705-4713.

[1]	何思达, 陈平华. 基于意图的轻量级自注意力序列推荐模型[J]. 计算机与现代化, 2024, 0(12): 1-9.
[2]	赵晨阳, 薛涛, 刘俊华. 基于改进Stable Diffusion的时尚服饰图案生成[J]. 计算机与现代化, 2024, 0(12): 15-23.
[3]	黄庭培1, 马禄彪1, 李世宝2, 刘建航1. 基于WiFi和原型网络的手势识别方法[J]. 计算机与现代化, 2024, 0(12): 34-39.
[4]	张晓东1, 白广芝1, 李敏1, 李昊洋2. 基于经验小波变换的油气井产量预测模型 [J]. 计算机与现代化, 2024, 0(12): 53-58.
[5]	刘云海1, 冯广1, 吴晓婷2, 杨群2. 复杂施工场景下的安全帽佩戴检测算法[J]. 计算机与现代化, 2024, 0(12): 66-71.
[6]	谷岳, 邓松峰, 沈霁, 穆文涛, 赵恩棋. 基于改进YOLOv8的SAR舰船目标检测算法[J]. 计算机与现代化, 2024, 0(12): 78-83.
[7]	王艳媛, 茅正冲. 中英文场景文本图像的检测和识别算法[J]. 计算机与现代化, 2024, 0(12): 84-90.
[8]	李钧超1, 尤菲1, 张超2, 苏乐乐2, 龚龑2. 基于新型多目标浣熊优化算法的BiLSTM-Attention#br# 预测模型及误差分析[J]. 计算机与现代化, 2024, 0(11): 70-76.
[9]	张宇1, 2, 黎靖1, 2, 马铭1, 2, 王众祥1, 2, 孙妍1, 2. YOLOLW:一个新的轻量级目标检测模型[J]. 计算机与现代化, 2024, 0(11): 91-98.
[10]	祁贤, 刘大铭, 常佳鑫. 基于改进自注意力机制的多视图三维重建[J]. 计算机与现代化, 2024, 0(11): 106-112.
[11]	杨骏1, 胡为1, 朱文福2. 基于改进MobileNetV3的视觉SLAM回环检测算法[J]. 计算机与现代化, 2024, 0(10): 21-26.
[12]	魏学诚1, 江凌云1, 李研2, 何非2. 改进YOLOv5的路侧单目视角小目标检测算法[J]. 计算机与现代化, 2024, 0(10): 27-34.
[13]	杜猛俊1, 李昂1, 童俊1, 钱锦1, 康恺1, 王若丁1, 靳文星2. 基于改进极限学习算法的电力信息数据融合模型[J]. 计算机与现代化, 2024, 0(10): 61-64.
[14]	杨世军1, 狄广义1, 高军1, 陈见飞1, 王耀坤1, 季晓晗2. 跨模态注意力融合和信息感知的情感一致检测[J]. 计算机与现代化, 2024, 0(10): 113-119.
[15]	候聪颖, 杨文清, 王召, 程聪. 基于时频自注意力残差时序卷积网络的语音增强[J]. 计算机与现代化, 2024, 0(09): 20-24.

基于改进YOLOX和新型数据关联方式的无人机#br# 多目标跟踪方法

Multi-object Tracking of UAV Based on Improved YOLOX and New Data Association Method

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价