基于CPN网络的车辆关键点检测

摘要/Abstract

摘要： 针对智慧交通和自动驾驶系统中利用车辆关键点获取车辆3D姿态的需求，提出一种基于CPN网络的车辆关键点检测模型。模型以ResNet50为主干网络，利用U型结构融合深层的语义信息和浅层的空间分辨率信息构建高斯热力图金子塔，随后采用SoftArgmax在高斯热力图中端到端地解码关键点坐标。在20万张的训练集上训练得到车辆关键点检测模型，模型能够同时预测定义的轿车和SUV车型上的78个关键点的坐标和可见性。经测试，在256×256的输入图下预测点的归一化像素误差为1.57，点的可见性预测在召回率为0.95时达到0.96的精确度。实验结果表明基于CPN网络的车辆关键点检测模型精度高，目前已应用于北京、武汉等城市的智慧交通系统中。

关键词: 智慧交通, 车辆关键点检测, CPN网络

Abstract: Aiming at the need to use vehicle key points to obtain vehicle 3D posture in smart transportation and self-driving systems, a vehicle key point detection model based on CPN network is proposed. The model integrates deep semantic information and shallow spatial resolution information in the form of a U-type structure with ResNet50 as the backbone network to build a Gaussian heat map pyramid. Then, SoftArgmax is used to decode the key point coordinates from the Gaussian heat map end to the end. The vehicle key point detection model is trained on a training set of 200000 sheets, which can predict the coordinates and visibility of 78 key points on defined cars and SUV models at the same time. The normalized pixel error of the prediction point under the input image of 256×256 is 1.57, and the visibility prediction of the point reaches the accuracy of 0.96 at the recall rate of 0.95. The experimental results show that the vehicle key point detection model based on the CPN network has high accuracy and has been applied to intelligent transportation systems in Beijing, Wuhan and other cities.

Key words: smart transportation, vehicle key point detection, CPN network

张志刚, 游安清. 基于CPN网络的车辆关键点检测[J]. 计算机与现代化, 2021, 0(10): 75-80.

ZHANG Zhi-gang, YOU An-qing. Vehicle Key Point Detection Based on CPN Network[J]. Computer and Modernization, 2021, 0(10): 75-80.

参考文献［26］

［1］	冯国臣. 基于智慧道路的车路协同应用研究［C］// 第十五届中国智能交通年会科技论文集（2）. 中国智能交通协会, 2020:652-661.
［2］	LIANG M, YANG B, WANG S L, et al. Deep continuous fusion for multi-sensor 3D object detection［C］// Proceedings of the 2018 European Conference on Computer Vision (ECCV). 2018:663-678.
［3］	CHEN X Z, KUNDU K, ZHANG Z Y, et al. Monocular 3D object detection for autonomous driving［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:2147-2156.
［4］	TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. 2015:4489-4497.
［5］	LEPETIT V, MORENO-NOGUER F, FUA P. EPnP: An accurate O(n) solution to the PnP problem［J］. International Journal of Computer Vision, 2009,81(2):155-166.
［6］	BHARDWAJ R, TUMMALA G K, RAMALINGAM G, et al. Autocalib: Automatic traffic camera calibration at scale［C］// Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments. 2017. DOI: 10.1145/3137133.3137149.
［7］	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM, 2017,60(6):84-90.
［8］	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition［J］. arXiv preprint arXiv:1409.1556, 2014.
［9］	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015:1-9.
［10］	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［11］	FENG Z H, KITTLER J, AWAIS M, et al. Wing loss for robust facial landmark localisation with convolutional neural networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:2235-2245.
［12］	FUHL W, KUBLER T, LOTZ R A, et al. Validation loss for landmark detection［J］. arXiv preprint arXiv:1901.10143, 2019.
［13］	NEWELL A, YANG K Y, DENG J. Stacked hourglass networks for human pose estimation［C］// Proceedings of the 2016 European Conference on Computer Vision (ECCV). 2016:483-499.
［14］	CAO Z, HIDALGO G, SIMON T, et al. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(1):172-186.
［15］	CHEN Y L, WANG Z C, PENG Y X, et al. Cascaded pyramid network for multi-person pose estimation［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018:7103-7112.
［16］	LI W B, WANG Z C, YIN B Y, et al. Rethinking on multi-stage networks for human pose estimation［J］. arXiv preprint arXiv:1901.00148, 2019.
［17］	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015:234-241.
［18］	HUANG J J, ZHU Z, GUO F, et al. The devil is in the details: Delving into unbiased data processing for human pose estimation［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020:5699-5708.
［19］	ZHANG F, ZHU X T, DAI H B, et al. Distribution-aware coordinate representation for human pose estimation［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020:7091-7100.
［20］	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6):1137-1149.
［21］	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017:936-944.
［22］	NIBALI A, HE Z, MORGAN S, et al. Numerical coordinate regression with convolutional neural networks［J］. arXiv preprint arXiv:1801.07372, 2018.
［23］	SONG X B, WANG P, ZHOU D F, et al. ApolloCar3D: A large 3D car instance understanding benchmark for autonomous driving［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019:5447-5457.
［24］	GOYAL P, DOLLAR P, GIRSHICK R, et al. Accurate, large minibatch SGD: Training imageNet in 1 hour［J］. arXiv preprint arXiv:1706.02677, 2017.
［25］	SUN K, XIAO B, LIU D, et al. Deep high-resolution representation learning for human pose estimation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019:5686-5696.
［26］	YU C Q, WANG J B, PENG C, et al. BiSeNet: Bilateral segmentation network for real-time semantic segmentation［C］// Proceedings of the 2018 European Conference on Computer Vision (ECCV). 2018:334-349.

[1]	郭之栋, 刘筠, 杨凡, 周昕, 陈亮亮. 基于机器视觉的智慧交通灯控制系统[J]. 计算机与现代化, 2023, 0(04): 101-105.
[2]	吴建波, 朱文霞, 剧亮, 许致芳. 边缘计算在智慧交通系统中的应用[J]. 计算机与现代化, 2021, 0(12): 103-109.