Driver Distracted Behavior Recognition Based on Deep Learning

Abstract

Abstract: Distracted driving behavior recognition is one of the main methods to improve driving safety. Aiming at the problem of low identification accuracy of distracted driving behavior, this paper proposes a driver distracted behavior recognition algorithm based on deep learning, which is composed of a cascade of target detection network and precise behavior recognition network. Based on the State Farm open data set, in the first level, the target detection algorithm SSD (Single Shot Multibox Detector) is used to extract local information from the original driver images in the data set and determine the candidate regions for behavior recognition. Then in the second level, the transfer learning VGG19, ResNet50 and MobileNetV2 models is used to accuratelyidentify the behavior information in the candidate region. Finally, the experiment compares the recognition accuracy of distracted driving behavior between layered recognition architecture and single model architecture. Results show that compared the proposed cascade network model with the mainstream model of single detection method, the driver behavior identification accuracy is improved 4% ~ 7% overall. Besides, the proposed algorithm not only reduces the influence of noise and other background regions on the model to improve the accuracy of distracted behavior recognition, but also can effectively identify more behavior categories to avoid the misclassification of actions.

Key words: driving safety, distracted driving behavior recognition, cascaded convolutional neural network model, transfer learning

HE Li-wen, ZHANG Rui-chi. Driver Distracted Behavior Recognition Based on Deep Learning[J]. Computer and Modernization, 2022, 0(06): 67-74.

References ［27］

［1］	NHTSA. Distracted driving ［EB/OL］. (2016-10-09)［2021-05-17］. https://www.nhtsa.gov/risky-driving/distracted-driving.
［2］	周欣,刘硕迪,潘薇,等. 自然交通场景中的车辆颜色识别［J］. 计算机科学, 2021,48(S1):15-20.
［3］	KUTILA M H, JOKELA M, MAKINEN T, et al. Driver cognitive distraction detection: Feature estimation and implementation［J］. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 2007,221(9):1027-1040.
［4］	OHN-BAR E, MARTIN S, TAWARI A, et al. Head, eye, and hand patterns for driver activity recognition［C］// 2014 22nd International Conference on Pattern Recognition. 2014:660-665.
［5］	REZAEI M, KLETTE R. Look at the driver, look at the road: No distraction! No accident!［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:129-136.
［6］	KOHLMOEGEN J, DORNHEGE G, BRAUN M L, et al. Improving Human Performance in a Real Operating Environment through Real-Time Mental Workload Detection［M］. MIT Press, 2007:409-422.
［7］	ZANDER T O, KOTHE C. Towards passive brain-computer interfaces: Applying brain-computer interface technology to human-machine systems in general［J］. Journal of Neural Engineering, 2011,8(2). DOI: 10.1088/1741-2560/8/2/025005.
［8］	WANG Y K, CHEN S A, LIN C T. An EEG-based brain-computer interface for dual task driving detection［J］. Neurocomputing, 2014,129:85-93.
［9］	ZHANG C, WANG H, FU R R. Automated detection of driver fatigue based on entropy and complexity measures［J］. IEEE Transportation Intelligent Transportation Systems, 2014,15(1):168-177.
［10］	WANG S Y, ZHANG Y Q, WU C X, et al. Online prediction of driver distraction based on brain activity patterns［J］. IEEE Transactions on Intelligent Transportation Systems, 2015,16(1):136-150.
［11］	YAN C, COENEN F, ZHANG B L. Driving posture recognition by convolutional neural networks［J］. IET Computer Vision, 2016,10(2):103-114.
［12］	LE T H N, ZHENG Y T, ZHU C C, et al. Multiple scale faster-RCNN approach to driver’s cell-phone usage and hands on steering wheel detection［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2016:46-53.
［13］	TRAN D, DO H, LU J X, et al. Real-time detection of distracted driving using dual cameras［C］// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2020:2014-2019.
［14］	崔华.白杰.毕欣.等. 具有多个级联输出层的深度神经网络［J］. 同济大学学报(自然科学版), 2017,45(S1):19-23.
［15］	XIONG Q F, LIN J, YUE W, et al. A deep learning approach to driver distraction detection of using mobile phone［C］// 2019 IEEE Vehicle Power and Propulsion Conference (VPPC). 2019. DOI: 10.1109/VPPC46532.2019.8952474.
［16］	陈军,张黎,周博,等. 基于级联卷积神经网络的驾驶员分心驾驶行为检测［J］. 科学技术与工程, 2020,20(14):5702-5708.
［17］	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014:580-587.
［18］	GIRSHICK R. Fast R-CNN［C］// 2015 IEEE International Conference on Computer Vision (ICCV). 2015:1440-1448.
［19］	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6):1137-1149.
［20］	REDMON J, FARHADI A. YOLOv3: An incremental improvement［J］. arXiv preprint arXiv:1804.02767, 2018.
［21］	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector［C］// Proceedings of the 2016 European Conference on Computer Vision. 2016:21-37.
［22］	LI J L, HOU Q S, XING J S, et al. SSD object detection model based on multi-frequency feature theory［J］. IEEE Access, 2020,8:82294-82305.
［23］	ZHAI S P, SHANG D R, WANG S H, et al. DF-SSD: An improved SSD object detection algorithm based on denseNet and feature fusion［J］. IEEE Access, 2020,8:24344-24357.
［24］	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks［C］// Proceedings of the 2012 Conference and Workshop on Neural Information Processing Systems (NIPS). 2012:1097-1105.
［25］	HE K M, ZHANG X Y, REN S Q, et al. Deep Residual Learning for Image Recognition［C］// 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016:770-778.
［26］	HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications［J］. arXiv preprint arXiv: 1704.04861, 2017.
［27］	SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2018:4510-4520.

[1]	HU Chong-jia, LIU Jin-zhou, FANG Li. Unsupervised Domain Adaptation for Outdoor Point Cloud Semantic Segmentation [J]. Computer and Modernization, 2024, 0(01): 74-79.
[2]	ZHANG Zhi-xia, XIE Bao-qiang. Natural Gas Load Forecasting Based on FCGA-LSTM and Transfer Learning [J]. Computer and Modernization, 2023, 0(07): 7-12.
[3]	YANG Jun, WANG Jin-lin, NI Hong, SHENG Yi-qiang, . Dynamic Transfer Method Based on Sensitivity in Industrial Control Network Anomaly Detection [J]. Computer and Modernization, 2023, 0(05): 46-51.
[4]	CHEN Xiao-wen, SHI Hui. A Digital Watermarking Detection Model Based on DWT-SVD and Transfer Learning [J]. Computer and Modernization, 2023, 0(04): 111-117.
[5]	BAI Xu-guang, LIU Cheng-zhong, HAN Jun-ying, GAO Jia-meng, CHEN Jun-kang. Classification Method of Small Sample Apple Leaves Based on SE-ResNeXt [J]. Computer and Modernization, 2023, 0(01): 18-23.
[6]	XU Ya-jun, GUO En-hao, CHEN Lin, SI Cheng-ke. A Data-driven Deep Modulation Identification Method for RF Signals [J]. Computer and Modernization, 2022, 0(06): 80-86.
[7]	ZHANG Yun-yao, HUANG He-ming, ZHANG Hui-yun, . Speech Recognition in Complex Noise Environment [J]. Computer and Modernization, 2021, 0(09): 68-74.
[8]	SU Hao, DING Sheng, ZHANG Chao-hua, . Ship Object Detection in Any Direction at Sea Based on Active and Transfer Learning [J]. Computer and Modernization, 2021, 0(09): 21-30.
[9]	CHEN Xiang, ZOU Qing-nian, XIE Shao-yu, CHEN Cui-qiong. Identification of Platen Switch State Based on Transfer Learning Strategy [J]. Computer and Modernization, 2021, 0(05): 120-126.
[10]	ZHANG Meng-yu. Flower Recognition Based on ResNet and Attention Mechanism [J]. Computer and Modernization, 2021, 0(04): 61-67.
[11]	XIE Xing-yu1, ZHANG Ying-lu2. Predicting Student Grades Ranking Based on Improved TrAdaboost Algorithm [J]. Computer and Modernization, 2016, 0(2): 122-126.
[12]	XIE Jiang-long, LIU Bo. Transfer Learning Algorithm Based on Tensor Data [J]. Computer and Modernization, 2016, 0(2): 42-48.
[13]	WANG Xing, ZHU Weicong, ZHANG Hao, ZHANG Xiaoxiao. An Algorithm for Travel Route Planning Merging with Weather Monitoring and Nowcasting [J]. Computer and Modernization, 2015, 0(2): 25-.