Hand Hygiene Action Quality Assessment Based on Multi-source Action Information

doi:10.3969/j.issn.1006-2475.2023.12.015

Abstract

Abstract: Abstract： The research on hand hygiene action quality assessment plays a crucial role in the intervention and improvement of hand hygiene behaviors. To address this task， this paper takes two different types of hand hygiene action information from video data and differential image data as inputs， creating a hand hygiene action quality assessment method based on multi-source action information. The algorithm consists of an action segmentation module and an evaluation module. In the action segmentation module， the feature segments associated with each step are divided by the position index. In the evaluation module， the differential image features obtained by the inter-frame difference method and ResNet50 feature extractor are introduced to combine with the past method （combining the optical flow and the I3D feature information of RGB） to capture the subtle hand motion information. The feature segments obtained by borrowing the segmentation module are processed and input to the hand hygiene information decoder based on the cross-attention mechanism， and the comprehensive features that fuse the details of hand motion are obtained. Next， these features are used to calculate the evaluation score of each step， and finally the evaluation score of each step is added to obtain the final evaluation result. The algorithm is verified by using the public data set HHA300. In the evaluation task， the evaluation index ρ and R-[ℓ]2（×100）achieves 0.86 and 0.95 respectively， which fully proves that the algorithm can accurately evaluate the motion quality of hand hygiene.

Key words: Key words： hand hygiene, action quality assessment, difference images, frame difference method, action segmentation, cross-attention

CLC Number:

TP391

LI De-kang, TANG Jin, WANG Fu-tian, TU Zi-jian, . Hand Hygiene Action Quality Assessment Based on Multi-source Action Information[J]. Computer and Modernization, 2023, 0(12): 87-93.

References

［1］张园. 扎根理论视角下的公立医院感染管理影响因素研究——以Z三甲医院为例［D］. 蚌埠：安徽财经大学， 2020.
［2］ BEARMAN G， DOLL M， COOPER K， et al. Hospital infection prevention： How much can we prevent and how hard should we try?［J］. Current Infectious Disease Reports， 2019，21（1）. DOI： 10.1007/s11908-019-0660-2.
［3］李光香，程伟荣. 六步洗手法对降低手细菌污染的临床观察［J］. 中国护理管理， 2005，5（1）：55-56.
［4］ LLORCA D F， PARRA I， SOTELO M A， et al. A vision-based system for automatic hand washing quality assessment［J］. Machine Vision and Applications， 2011，22（2）：219-234.
［5］ ZHONG C Z， REIBMAN A R， MINA H A， et al. Designing a computer-vision application： A case study for hand-hygiene assessment in an open-room environment［J］. Journal of Imaging， 2021，7（9）. DOI： 10.3390/jimaging7090170.
［6］ LI C L， ZHU Q W， LIU T B， et al. Hand hygiene assessment via joint step segmentation and key action scorer［J］. arXiv preprint arXiv：2209.12221， 2022.
［7］ CARREIRA J， ZISSERMAN A. Quo vadis， action recognition? A new model and the kinetics dataset［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：4724-4733.
［8］ BAI S J， KOLTER J Z， KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling［J］. arXiv preprint arXiv：1803.01271， 2018.
［9］ YU F， KOLTUN V. Multi-scale context aggregation by dilated convolutions［J］. arXiv preprint arXiv：1511.07122， 2015.
［10］ OORD A V D， DIELEMAN S， ZEN H， et al. WaveNet： A generative model for raw audio［J］. arXiv preprint arXiv：1609.03499， 2016.
［11］ LEA C， FLYNN M D， VIDAL R， et al. Temporal convolutional networks for action segmentation and detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2017：1003-1012.
［12］ LEI P， TODOROVIC S. Temporal deformable residual networks for action segmentation in videos［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2018：6742-6751.
［13］ FARHA Y A， GALL J. MS-TCN： Multi-stage temporal convolutional network for action segmentation［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2019：3570-3579.
［14］ ISHIKAWA Y， KASAI S， AOKI Y， et al. Alleviating over-segmentation errors by detecting action boundaries［C］// Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision （WACV）. 2021：2321-2330.
［15］ WANG Z Z， GAO Z T， WANG L M， et al. Boundary-aware cascade networks for temporal action segmentation［C］// Proceedings of the 2020 European Conference on Computer Vision. 2020：34-51.
［16］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017：6000-6010.
［17］ YI F Q， WEN H Y， JIANG T T. ASFormer： Transformer for action segmentation［J］. arXiv preprint arXiv：2110.08568， 2021.
［18］ PIRSIAVASH H， VONDRICK C， TORRALBA A. Assessing the quality of actions［C］// Proceedings of the 2014 European Conference on Computer Vision. 2014：556-571.
［19］ LI Z Q， HUANG Y F， CAI M J， et al. Manipulation-skill assessment from videos with spatial attention network［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop （ICCVW）. 2019：4385-4395.
［20］ PARMAR P， TRAN MORRIS B. Learning to score olympic events［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops （CVPRW）. 2017：76-84.
［21］ TRAN D， BOURDEV L， FERGUS R， et al. Learning spatiotemporal features with 3D convolutional networks［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision （ICCV）. 2015：4489-4497.
［22］ ZENG L A， HONG F T， ZHENG W S， et al. Hybrid dynamic-static context-aware attention network for action assessment in long videos［C］// Proceedings of the 28th ACM International Conference on Multimedia. 2020：2526-2534
［23］ YU X M， RAO Y M， ZHAO W L， et al. Group-aware contrastive regression for action quality assessment［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision （ICCV）. 2021：7919-7928.
［24］ XU C M， FU Y W， ZHANG B， et al. Learning to score figure skating sport videos［J］. IEEE Transactions on Circuits and Systems for Video Technology， 2020，30（12）：4578-4590.
［25］ XU A C， ZENG L A， ZHENG W S. Likert scoring with grade decoupling for long-term action assessment［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）. 2022：3222-3231.
［26］ LIKERT R. A technique for the measurement of attitudes［J］. Archives of Psychology， 1932.
［27］ HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）. 2016：770-778.

[1]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[2]	YANG Shu-guo1, HE Wen-jing1, LIU Yin-ling1, MA Zuo-lin1, HU Shuai2. #br# A Moving Target Algorithm Based on Gaussian Mixture Model #br# and Eight-neighbor FDM with Morphological Processing [J]. Computer and Modernization, 2019, 0(07): 32-.
[3]	ZHANG Wei-ming, ZHOU Wu-neng. An Improved ViBe Algorithm Based on Ghost Judgment #br# Suppression and Local Motion Compensation [J]. Computer and Modernization, 2018, 0(01): 51-55.