计算机与现代化 ›› 2023, Vol. 0 ›› Issue (12): 87-93.doi: 10.3969/j.issn.1006-2475.2023.12.015

• 图像处理 • 上一篇    下一篇

基于多源动作信息的手卫生动作质量评估

  

  1. (1.安徽医科大学生物医学工程学院,安徽 合肥 230032; 2.合肥综合性国家科学中心人工智能研究院,安徽 合肥 230088;
    3.安徽大学,安徽 合肥 230039)
  • 出版日期:2023-12-24 发布日期:2024-01-29
  • 作者简介:李德康(1998—),男,安徽宿州人,硕士研究生,研究方向:计算机视觉,手卫生动作质量评估,E-mail: 1368274643@qq. com; 通信作者:汤进(1976—),男,安徽合肥人,教授,博士,研究方向:计算机视觉与深度学习,医学图像处理,E-mail: tangjin@ahu.edu.cn。
  • 基金资助:
    国家自然科学基金资助项目(62076003); 合肥市自然科学基金资助项目(HZ22ZK001)

Hand Hygiene Action Quality Assessment Based on Multi-source Action Information

  1. (1. School of Biomedical Engineering, Anhui Medical University, Hefei 230032, China;
    2. Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei 230088, China;
    3. Anhui University, Hefei 230039, China)
  • Online:2023-12-24 Published:2024-01-29

摘要: 摘要:手卫生动作质量评估任务的研究对于手卫生行为的干预可以发挥重要作用。为此,本文将来自视频数据和差分图像数据的2种不同类型的手卫生动作信息作为输入,构建一种基于多源动作信息的手卫生动作质量评估方法。该算法包含动作分割模块和评估模块,在动作分割模块中,通过位置索引划分与每个步骤相关的特征片段。在评估模块中,引入通过帧间差分法和ResNet50特征提取器获取的差分图像特征,与过去的方法(结合光流和RGB的I3D特征信息)相结合,以捕捉手部细微动作信息。将分割模块获取的特征片段经过处理输入基于交叉注意力机制的手卫生信息解码器,得到融合手部运动细节信息的综合特征。使用这些特征计算每个步骤的评估得分,最后将各步骤的评估得分相加以获得最终评估结果。该算法通过使用来自公开数据集HHA300进行验证,在评估任务中,评价指标ρ、R-[ℓ]2(×100)分别取得了0.86和0.95的结果,充分说明了该算法能够准确评估手卫生的动作质量。

关键词: 关键词:手卫生, 动作质量评估, 差分图像, 帧间差分法, 动作分割, 交叉注意力

Abstract: Abstract: The research on hand hygiene action quality assessment plays a crucial role in the intervention and improvement of hand hygiene behaviors. To address this task, this paper takes two different types of hand hygiene action information from video data and differential image data as inputs, creating a hand hygiene action quality assessment method based on multi-source action information. The algorithm consists of an action segmentation module and an evaluation module. In the action segmentation module, the feature segments associated with each step are divided by the position index. In the evaluation module, the differential image features obtained by the inter-frame difference method and ResNet50 feature extractor are introduced to combine with the past method (combining the optical flow and the I3D feature information of RGB) to capture the subtle hand motion information. The feature segments obtained by borrowing the segmentation module are processed and input to the hand hygiene information decoder based on the cross-attention mechanism, and the comprehensive features that fuse the details of hand motion are obtained. Next, these features are used to calculate the evaluation score of each step, and finally the evaluation score of each step is added to obtain the final evaluation result. The algorithm is verified by using the public data set HHA300. In the evaluation task, the evaluation index  ρ and  R-[ℓ]2(×100)achieves 0.86 and 0.95 respectively, which fully proves that the algorithm can accurately evaluate the motion quality of hand hygiene.

Key words: Key words: hand hygiene, action quality assessment, difference images, frame difference method, action segmentation, cross-attention

中图分类号: