Behavior Recognition Method Based on FMCW Radar and ResNeSt-GRU

doi:10.3969/j.issn.1006-2475.2023.11.016

Abstract

Abstract: Abstract： Aiming at the application of frequency modulated continuous wave radar in behavior recognition， a human behavior recognition system based on split attention residual neural network （ResNeSt） and gated neural unit （GRU） is proposed. The frequency modulated continuous wave （FMCW） radar is used to collect human behavior data. The fast Fourier transform algorithm （FFT） is used to extract the distance， velocity and angle dimension information of each frame of radar data， and then stitch them according to the time dimension into Range-Time Map （RTM）， Doppler-Time Map （DTM） and Angle-Time Map （ATM）. Finally， RTM， DTM and ATM are used as input samples， and the three-stream ResNeSt-GRU model is used to recognize different human behaviors. The experimental results show that the average recognition accuracy of the three-stream ResNeSt-GRU model for 8 behaviors reaches 98.92%， which is higher than the traditional deep learning model and the fusion deep learning model. In addition， the recognition accuracy rate using this model is 2.3% higher than that using a single-stream network after traditional feature fusion. Therefore， the system can effectively improve the recognition accuracy of the human behavior recognition system， and provide a new technology for the human behavior recognition.

Key words: Key words： ResNeSt, GRU, FMCW radar, deeping learning, behavior recognition

CLC Number:

TP391.4

MA Ze-yu, YE Ning, XU Kang, WANG Su, WANG Ru-chuan, . Behavior Recognition Method Based on FMCW Radar and ResNeSt-GRU[J]. Computer and Modernization, 2023, 0(11): 101-107.

References ［28］

［1］	MAJUMDER A， ZERIN I， AHAMED S I， et al. A multi-sensor approach for fall risk prediction and prevention in elderly［J］. ACM SIGAPP Applied Computing Review， 2014，14（1）：41-52.
［2］	SATHYANARAYANA A， JOTY S， FERNANDEZ-LUQUE
	L， et al. Correction of： Sleep quality prediction from wearable data using deep learning［J］. JMIR Mhealth and Uhealth， 2016，4（4）：e130. DOI： 10.2196/mhealth.6562.
［3］	WANG Z L， WU D H， CHEN J M， et al. A triaxial accelerometer-based human activity recognition via EEMD-
	Based features and game-theory-based feature selection［J］. IEEE Sensors Journal， 2016，16（9）：3198-3207.
［4］	ALARIFI A， ALWADAIN A. Killer heuristic optimized convolution neural network-based fall detection with wearable IoT sensor devices［J］. Measurement， 2020，167：108258.
	1-108258.10.
［5］	KOSUGE A， SUEHIRO S， HAMADA M， et al. mmWave-YOLO： A mmWave imaging radar-based real-time multiclass object recognition system for ADAS applications［J］. IEEE Transactions on Instrumentation and Measurement， 2022，71：2509810.1-2509810.10.
［6］	YANG X， LIU J， CHEN Y Y， et al. MU-ID： Multi-user identification through gaits using millimeter wave radios［C］// IEEE Conference on Computer Communications. IEEE， 2020： 2589-2598.
［7］	宋文豪，周斌，卜智勇，等. 基于复合型FMCW波形的多目标匹配算法［J］. 中国科学院大学学报， 2022，39（1）：110-118.
［8］	RAEIS H， KAZEMI M， SHIRMOHAMMADI S. InARMS： Individual activity recognition of multiple subjects with FMCW radar［C］// 2022 IEEE International Instrumentation and Measurement Technology Conference（I2MTC）. IEEE， 2022： 1-5.
［9］	许志猛，尹辉斌，林佳慧. 基于空间聚类的FMCW雷达双人行为识别方法［J］. 福州大学学报：自然科学版， 2020，48（4）：445-450.
［10］	WANG S W，SONG J，LIEN J，et al．Interacting with soli： Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum［C］// Symposium on User Interface Software and Technology. ACM， 2016：851-860.
［11］	王俊，郑彤，雷鹏，等. 基于卷积神经网络的手势动作雷达识别方法［J］. 北京航空航天大学学报， 2018，44（6）：1117-1123.
［12］	AHMAD A， ROH J C， WANG D， et al. Vital signs monitoring of multiple people using a FMCW millimeter-wave sensor［C］// IEEE Radar Conference（RadarConf18）. IEEE， 2018：1450-1455.
［13］	SENIGAGLIESI L， CIATTAGLIA G， GAMBI E. Contactless walking recognition based on mmWave RADAR［C］// 2020 IEEE Symposium on Computers and Communications（ISCC）. IEEE， 2020：1-4.
［14］	张宇成，陈金立. 基于RDATM数据集的毫米波雷达动态手势识别系统设计［J］. 中国电子科学研究院学报， 2022，17（5）：457-464.
［15］	ZHENG Z J， PAN J， NI Z K， et al. Human posture reconstruction for through-the-wall radar imaging using convolutional neural networks［J］. IEEE Geoscience and Remote Sensing Letters， 2022， 19：1-5.
［16］	元志安，周笑宇，刘心溥，等. 基于RDSNet的毫米波雷达人体跌倒检测方法［J］. 雷达学报， 2021，10（4）：656-664.
［17］	ALIREZAZAD K， RHIEL G， MAURER L. 2D CNN-GRU model for multi-hand gesture recognition system using FMCW Radar［C］// Proceedings of the 20th IEEE Interregional NEWCAS Conference（NEWCAS）. IEEE， 2022：158-162.
［18］	王昊飞，李俊峰. 基于注意力机制的改进残差网络的人体行为识别方法［J］. 软件工程， 2021，24（11）：51-54.
［19］	SCHEIBLHOFER W， FEGER R， HADERER A， et al. Method to embed a data-link on FMCW chirps for communication between cooperative 77-GHz radar stations［C］// 2015 European Radar Conference（EuRAD）. IEEE， 2015：181-184.
［20］	PENG Z Y， MUNOZ-FERRERAS J M， GOMEZ-GARCIA
	R， et al. FMCW radar fall detection based on ISAR processing utilizing the properties of RCS， range， and Doppler［C］// IEEE MTT-S International Microwave Symposium. IEEE， 2016：1-3.
［21］	AHEMD S， KIM W， PARK J， et al. Radar-based air-writing gesture recognition using a novel multistream CNN approach［J］. IEEE Internet of Things Journal， 2022，9（23）：23869-23880.
［22］	ENGGAR F D， MUTHIAH A M， WINARKO O D， et al. Performance comparison of various windowing on FMCW radar signal processing［C］// 2016 International Symposium on Electronics and Smart Devices（ISESD）. IEEE， 2016：326-330.
［23］	ZHANG H， WU C R， ZHANG Z Y， et al. ResNeSt： Split-attention networks［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops（CVPRW）.
	IEEE， 2022：2735-2745.
［24］	XIE S N， GIRSHICK R， DOLLÁR P， et al. Aggregated residual transformations for deep neural networks［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）. IEEE， 2017：5987-5995.
［25］	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. IEEE， 2018：7132-7141.
［26］	LI X， WANG W H， HU X L， et al. Selective kernel networks［C］// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）. IEEE， 2019：510-519.
［27］	TODERICI G， VINCENT D， JOHNSTON N， et al. Full resolution image compression with recurrent neural networks［C］// 2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）. IEEE， 2017：5435-5443.
［28］	BADLANI R， ŁANCUCKI A， SHIH K J， et al. One TTS alignment to rule them all［C］// IEEE International Conference on Acoustics， Speech and Signal Processing（ICASSP）.
	IEEE， 2022：6092-6096.

[1]	ZHOU Hui, XU Ming-hai, XU Xiao-dong. Chinese Word Segmentation Model Based on Attention-BIGRU-CRF [J]. Computer and Modernization, 2022, 0(08): 7-12.
[2]	PENG Peng-fei, ZHOU Lin-ru. GRU Adversarial Network Text Generation Model with Reward [J]. Computer and Modernization, 2022, 0(07): 121-126.
[3]	FAN Hai-wei, QIN Jia-jie, SUN Huan, ZHANG Li-miao, LU Xin-siyu. Traffic Accident Text Information Extraction Model Based on BERT and BiGRU-CRF Fusion [J]. Computer and Modernization, 2022, 0(05): 10-15.
[4]	ZHAO Yu-xuan, HU Huai-xiang. Spam Recognition Method Based on BiGRU-Attention-CNN Model [J]. Computer and Modernization, 2021, 0(04): 122-126.
[5]	SUN Li-li, GUO Lin, WEN Xu, ZHANG Wen-nuo. Rural Novel Translation Methods Based on Bidirectional GRU Neural Network Machine Model [J]. Computer and Modernization, 2021, 0(04): 27-31.
[6]	Nigara Masimjan, Azragul Yusuf. Analysis of Emotional Tendency of MOOC User Comments Based on BERT and Bidirectional GRU Model#br# [J]. Computer and Modernization, 2021, 0(04): 20-26.
[7]	FENG Yun-xia, YI Peng, HANG Zheng-liang, SONG Bo. Entity Extraction Method of Chinese Electronic Medical Record Based on CNN-BGRU-CRF [J]. Computer and Modernization, 2020, 0(11): 60-64.
[8]	GAO Bai-hong1, LIU Zhao-hui1, LIU Hua2. Network Traffic Prediction Based on SCSO-GRU Model [J]. Computer and Modernization, 2020, 0(04): 72-.
[9]	LYU Tian. A Multi-features Short-term Traffic Flow Prediction Method Based on SDZ-GRU [J]. Computer and Modernization, 2019, 0(10): 60-.
[10]	TANG Kun1,2, CHEN Si-si1,3. GRU and LDA Based Group Chat Topic Mining [J]. Computer and Modernization, 2018, 0(12): 72-.
[11]	LI Lei， CHEN Ai-xiang， LI Wei-shu， LIANG Wei-qi， YANG Si-tong. GRU Neural Network’s Prediction of Stock Closing Price [J]. Computer and Modernization, 2018, 0(11): 103-.
[12]	YE Xiao-zhou, TAO Fei-fei, QI Rong-zhi, ZHANG Yun-fei, ZHOU Si-qi, LIU Xuan. Improvement on Activation Functions of Recurrent Neural Network Architectures [J]. Computer and Modernization, 2016, 0(12): 29-33.
[13]	LI Ai-hua, WANG Li-bin. An Image Quality Assessment Algorithm Based on Phase Feature for Wavelet Domain [J]. Computer and Modernization, 2015, 0(8): 112-115.
[14]	LU You-bing;GENG Huan-tong;ZHANG Ming;LIU Jing-fa. Solving Congruent Circle Packing Problem Based on Improved Differential Evolution Algorithm [J]. Computer and Modernization, 2012, 1(6): 1-4,22.
[15]	ZHANG Renlong;ZHOU Qiyun;HU Bin;JIANG Jiebao;WEN Shuisheng. Research and Implementation of Test Paper Intellectual Difficulty Adaptively Generating [J]. Computer and Modernization, 2012, 1(03): 8-10.