基于深度学习的驾驶员分心行为识别

摘要/Abstract

摘要： 分心驾驶行为识别是提高驾驶安全的主要方法之一。针对分心驾驶行为识别精度低的问题，本文提出一种基于深度学习的驾驶员分心行为识别算法，由目标检测网络和行为精确识别网络级联构成。基于State Farm公开数据集，第一级利用目标检测算法SSD(Single Shot Multibox Detector)对数据集中的驾驶员原始图像进行局部信息提取，确定行为识别候选区域；第二级分别利用迁移学习VGG19、ResNet50和MobileNetV2模型对候选区域内的行为信息进行精确识别；最后，实验对比级联架构与单模型架构对分心驾驶行为的识别精度。结果表明，提出的级联网络模型相较于主流单模型检测方法，驾驶员行为识别的准确率总体上提升4~7%个百分点。该算法不仅减少噪声和其他背景区域对模型的影响，提高分心行为识别准确率，还可以有效识别更多的行为类别以避免动作的误分类。

关键词: 驾驶安全, 分心驾驶行为识别, 级联卷积神经网络模型, 迁移学习

Abstract: Distracted driving behavior recognition is one of the main methods to improve driving safety. Aiming at the problem of low identification accuracy of distracted driving behavior, this paper proposes a driver distracted behavior recognition algorithm based on deep learning, which is composed of a cascade of target detection network and precise behavior recognition network. Based on the State Farm open data set, in the first level, the target detection algorithm SSD (Single Shot Multibox Detector) is used to extract local information from the original driver images in the data set and determine the candidate regions for behavior recognition. Then in the second level, the transfer learning VGG19, ResNet50 and MobileNetV2 models is used to accuratelyidentify the behavior information in the candidate region. Finally, the experiment compares the recognition accuracy of distracted driving behavior between layered recognition architecture and single model architecture. Results show that compared the proposed cascade network model with the mainstream model of single detection method, the driver behavior identification accuracy is improved 4% ~ 7% overall. Besides, the proposed algorithm not only reduces the influence of noise and other background regions on the model to improve the accuracy of distracted behavior recognition, but also can effectively identify more behavior categories to avoid the misclassification of actions.

Key words: driving safety, distracted driving behavior recognition, cascaded convolutional neural network model, transfer learning

何丽雯, 张锐驰. 基于深度学习的驾驶员分心行为识别[J]. 计算机与现代化, 2022, 0(06): 67-74.

HE Li-wen, ZHANG Rui-chi. Driver Distracted Behavior Recognition Based on Deep Learning[J]. Computer and Modernization, 2022, 0(06): 67-74.

参考文献［27］

［1］	NHTSA. Distracted driving ［EB/OL］. (2016-10-09)［2021-05-17］. https://www.nhtsa.gov/risky-driving/distracted-driving.
［2］	周欣,刘硕迪,潘薇,等. 自然交通场景中的车辆颜色识别［J］. 计算机科学, 2021,48(S1):15-20.
［3］	KUTILA M H, JOKELA M, MAKINEN T, et al. Driver cognitive distraction detection: Feature estimation and implementation［J］. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2007,221(9):1027-1040.
［4］	OHN-BAR E, MARTIN S, TAWARI A, et al. Head, eye, and hand patterns for driver activity recognition［C］// 2014 22nd International Conference on Pattern Recognition. 2014:660-665.
［5］	REZAEI M, KLETTE R. Look at the driver, look at the road: No distraction! No accident!［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:129-136.
［6］	KOHLMOEGEN J, DORNHEGE G, BRAUN M L, et al. Improving Human Performance in a Real Operating Environment through Real-Time Mental Workload Detection［M］. MIT Press, 2007:409-422.
［7］	ZANDER T O, KOTHE C. Towards passive brain-computer interfaces: Applying brain-computer interface technology to human-machine systems in general［J］. Journal of Neural Engineering, 2011,8(2). DOI: 10.1088/1741-2560/8/2/025005.
［8］	WANG Y K, CHEN S A, LIN C T. An EEG-based brain-computer interface for dual task driving detection［J］. Neurocomputing, 2014,129:85-93.
［9］	ZHANG C, WANG H, FU R R. Automated detection of driver fatigue based on entropy and complexity measures［J］. IEEE Transportation Intelligent Transportation Systems, 2014,15(1):168-177.
［10］	WANG S Y, ZHANG Y Q, WU C X, et al. Online prediction of driver distraction based on brain activity patterns［J］. IEEE Transactions on Intelligent Transportation Systems, 2015,16(1):136-150.
［11］	YAN C, COENEN F, ZHANG B L. Driving posture recognition by convolutional neural networks［J］. IET Computer Vision, 2016,10(2):103-114.
［12］	LE T H N, ZHENG Y T, ZHU C C, et al. Multiple scale faster-RCNN approach to driver’s cell-phone usage and hands on steering wheel detection［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2016:46-53.
［13］	TRAN D, DO H, LU J X, et al. Real-time detection of distracted driving using dual cameras［C］// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2020:2014-2019.
［14］	崔华.白杰.毕欣.等. 具有多个级联输出层的深度神经网络［J］. 同济大学学报(自然科学版), 2017,45(S1):19-23.
［15］	XIONG Q F, LIN J, YUE W, et al. A deep learning approach to driver distraction detection of using mobile phone［C］// 2019 IEEE Vehicle Power and Propulsion Conference (VPPC). 2019. DOI: 10.1109/VPPC46532.2019.8952474.
［16］	陈军,张黎,周博,等. 基于级联卷积神经网络的驾驶员分心驾驶行为检测［J］. 科学技术与工程, 2020,20(14):5702-5708.
［17］	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:580-587.
［18］	GIRSHICK R. Fast R-CNN［C］// 2015 IEEE International Conference on Computer Vision (ICCV). 2015:1440-1448.
［19］	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6):1137-1149.
［20］	REDMON J, FARHADI A. YOLOv3: An incremental improvement［J］. arXiv preprint arXiv:1804.02767, 2018.
［21］	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector［C］// Proceedings of the 2016 European Conference on Computer Vision. 2016:21-37.
［22］	LI J L, HOU Q S, XING J S, et al. SSD object detection model based on multi-frequency feature theory［J］. IEEE Access, 2020,8:82294-82305.
［23］	ZHAI S P, SHANG D R, WANG S H, et al. DF-SSD: An improved SSD object detection algorithm based on denseNet and feature fusion［J］. IEEE Access, 2020,8:24344-24357.
［24］	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks［C］// Proceedings of the 2012 Conference and Workshop on Neural Information Processing Systems (NIPS). 2012:1097-1105.
［25］	HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［26］	HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications［J］. arXiv preprint arXiv: 1704.04861, 2017.
［27］	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2018:4510-4520.

[1]	徐亚军, 郭恩豪, 陈林, 司成可. 数据驱动的RF信号深度调制识别方法[J]. 计算机与现代化, 2022, 0(06): 80-86.
[2]	冯敬翔. 基于迁移学习的卷积神经网络通道剪枝[J]. 计算机与现代化, 2021, 0(12): 13-18.
[3]	苏浩, 丁胜, 章超华, . 主动迁移学习的海上任意方向船只目标检测[J]. 计算机与现代化, 2021, 0(09): 21-30.
[4]	张允耀, 黄鹤鸣, 张会云, . 复杂噪声环境下语音识别研究[J]. 计算机与现代化, 2021, 0(09): 68-74.
[5]	陈翔, 邹庆年, 谢绍宇, 陈翠琼. 基于迁移学习策略的压板开关状态识别[J]. 计算机与现代化, 2021, 0(05): 120-126.
[6]	张梦雨. 基于ResNet和注意力机制的花卉识别[J]. 计算机与现代化, 2021, 0(04): 61-67.
[7]	徐龙. 基于自注意力和胶囊网络的短文本情感分析[J]. 计算机与现代化, 2020, 0(07): 61-64.
[8]	谢江龙，刘波. 面向张量数据的迁移学习算法[J]. 计算机与现代化, 2016, 0(2): 42-48.
[9]	谢星宇1,张颖璐2. 基于改进的TrAdaboost算法的学生成绩排名预测[J]. 计算机与现代化, 2016, 0(2): 122-126.