Micro-expression Recognition Based on AU-GCN and Attention Mechanism

Abstract

Abstract: As a kind of expression with very short duration， micro-expression can implicitly express people ’s true feelings of trying to suppress and hide， which has a good application in national security， judicial system， medical category and political elections. However， since micro-expression has the characteristics of less data sets， short duration and low expression amplitude， there are many difficulties in identifying micro-expressions， such as less data samples， larger calculation， lack of attention to key features， and easy to over-fitting. Therefore， this paper uses facial action units （ AU ） to highlight local features by weighted attention mechanism， and applies graph convolution network to find the dependencies between AU nodes， and aggregates them into global features for micro-expression recognition. The experimental results show that compared with the existing methods， the proposed method improves the accuracy to 79.3 %.

Key words: micro-expression, facial action unit, graph convolution network, attention mechanism

ZHAO Jing-hua, YANG Qiu-xiang. Micro-expression Recognition Based on AU-GCN and Attention Mechanism[J]. Computer and Modernization, 2023, 0(03): 48-53.

References

［1］刘振南，申寻兵. 微表情识别训练对大学生欺骗检测能力的作用研究［C］// 第23届全国心理学学术会议摘要集（下）. 2021：397-398.
［2］吴奇，申寻兵，傅小兰. 微表情研究及其应用［J］. 心理科学进展， 2010，18（9）：1359-1368.
［3］蔡村，陈正东，沈蓓蓓. 把握瞬间真实：海关旅检应用微表情心理学的构想［J］. 海关与经贸研究， 2018，39（3）：27-34.
［4］ SZAJNBERG N M. What the Face Reveals： Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System （FACS）［M］. Oxford University Press， 1997.
［5］ POLIKOVSKY S， KAMEDA Y， OHTA Y. Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor［C］// The 3rd International Conference on Imaging for Crime Detection and Prevention. 2009：1-6.
［6］陶西浩，万定生，杨志勇. 一种基于改进 LBP 特征的人脸检测方法［J］. 信息技术， 2018，42（2）：1-4.
［7］卢官明，杨成，杨文娟，等. 基于 LBP-TOP 特征的微表情识别［J］. 南京邮电大学学报（自然科学版）， 2017，37（6）：1-7.
［8］ SHREVE M， GODAVARTHY S， MANOHAR V， et al. Towards macro-and micro-expression spotting in video using strain patterns［C］// 2009 Workshop on Applications of Computer Vision （WACV）. 2009：1-6.
［9］ SHREVE M， GODAVARTHY S， GOLDGOF D， et al. Macro-and micro-expression spotting in long videos using spatio-temporal strain［C］// 2011 IEEE International Conference on Automatic Face & Gesture Recognition （FG）. 2011：51-56.
［10］ PFISTER T， LI X B， ZHAO G Y， et al. Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework［C］// 2011 IEEE International Conference on Computer Vision Workshops （ICCV Workshops）. 2011：868-875.
［11］ GUPTA P. MERASTC： Micro-expression recognition using effective feature encodings and 2D convolutional neural network［J］. IEEE Transactions on Affective Computing， 2021. DOI：10.1109/TAFFC.2021.3061967.
［12］ LO L， XIE H X， SHUAI H H， et al. MER-GCN： Micro-expression recognition based on relation modeling with graph convolutional networks［C］// 2020 IEEE Conference on Multimedia Information Processing and Retrieval （MIPR）. 2020：79-84.
［13］ XIE H X， LO L， SHUAI H H， et al. Au-assisted graph attention convolutional network for micro-expression recognition［C］// Proceedings of the 28th ACM International Conference on Multimedia. 2020：2871-2880.
［14］ BUHARI A M， OOI C P， BASKARAN V M， et al. FACS-based graph features for real-time micro-expression recognition［J］. Journal of Imaging， 2020，6（12）：130.
［15］ DAVISON A K， LANSLEY C， COSTEN N， et al. SAMM： A spontaneous micro-facial movement dataset［J］. IEEE Transactions on Affective Computing， 2016，9（1）：116-129.
［16］ KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks［J］. arXiv preprint arXiv：1609.02907， 2016.
［17］ ASTHANA A， ZAFEIRIOU S， CHENG S， et al. Robust discriminative response map fitting with constrained local models［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013：3444-3451.
［18］魏文辉. 基于代表性AU区域的微表情识别算法研究［D］. 济南：山东大学， 2021.
［19］张延良，卢冰，洪晓鹏，等. 基于局部区域方法的微表情识别［J］. 计算机应用， 2019，39（5）：1282-1287.
［20］何景琳，梁正友，孙宇，等. 结合C3D与光流法的微表情自动识别［J］. 计算机系统应用， 2021，30（1）：221-227.
［21］ VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Advances in Neural Information Processing Systems. 2017：5998-6008.
［22］ TRAN D， WANG H， TORRESANI L， et al. A closer look at spatiotemporal convolutions for action recognition［C］// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018：6450-6459.
［23］ LI J， WANG Y D， SEE J， et al. Micro-expression recognition based on 3D flow convolutional neural network［J］. Pattern Analysis and Applications， 2019，22（4）：1331-1339.
［24］ KHOR H Q， SEE J， LIONG S T， et al. Dual-stream shallow networks for facial micro-expression recognition［C］// 2019 IEEE International Conference on Image Processing （ICIP）. 2019：36-40.
［25］ LEI L， LI J F， CHEN T， et al. A novel graph-tcn with a graph structured representation for micro-expression recognition［C］// Proceedings of the 28th ACM International Conference on Multimedia. 2020：2237-2245.
［26］ XIA Z Q， HONG X P， GAO X Y， et al. Spatiotemporal recurrent convolutional networks for recognizing spontaneous micro-expressions［J］. IEEE Transactions on Multimedia， 2019，22（3）：626-640.

[1]	HE Sida, CHEN Pinghua. Intent-based Lightweight Self-Attention Network for Sequential Recommendation [J]. Computer and Modernization, 2024, 0(12): 1-9.
[2]	ZHAO Chenyang, XUE Tao, LIU Junhua. Fashion Clothing Pattern Generation Based on Improved Stable Diffusion [J]. Computer and Modernization, 2024, 0(12): 15-23.
[3]	HUANG Tingpei1, MA Lubiao1, LI Shibao2, LIU Jianhang1. Gesture Recognition Method Based on WiFi and Prototypical Network [J]. Computer and Modernization, 2024, 0(12): 34-39.
[4]	ZHANG Xiaodong1, BAI Guangzhi1, LI Min1, LI Haoyang2. Oil and Gas Well Production Prediction Model Based on Empirical Wavelet Transform [J]. Computer and Modernization, 2024, 0(12): 53-58.
[5]	WANG Yanyuan, MAO Zhengchong. Detection and Recognition Algorithms for Chinese and English Scene Text Images [J]. Computer and Modernization, 2024, 0(12): 84-90.
[6]	LI Junchao1, YOU Fei1, ZHANG Chao2, SU Lele2, GONG Yan2 . BiLSTM-Attention Prediction Model and Error Analysis #br# Based on Novel Multi-objective Coati Optimization Algorithm [J]. Computer and Modernization, 2024, 0(11): 70-76.
[7]	ZHANG Yu1, 2, LI Jing1, 2, MA Ming1, 2, WANG Zhongxiang1, 2, SUN Yan1, 2. YOLOLW: A Novel Lightweight Object Detection Model [J]. Computer and Modernization, 2024, 0(11): 91-98.
[8]	QI Xian, LIU Daming, CHANG Jiaxin. Multi-view 3D Reconstruction Based on Improved Self-attention Mechanism [J]. Computer and Modernization, 2024, 0(11): 106-112.
[9]	YANG Jun1, HU Wei1, ZHU Wenfu2. Visual SLAM Loop Closure Detection Algorithm Based on Improved MobileNetV3 [J]. Computer and Modernization, 2024, 0(10): 21-26.
[10]	WEI Xuecheng1, JIANG Lingyun1, LI Yan2, HE Fei2. Improved Roadside Monocular View Small Target Detection Algorithm Based on YOLOv5 [J]. Computer and Modernization, 2024, 0(10): 27-34.
[11]	DU Mengjun1, LI Ang1, TONG Jun1, QIAN Jin1, KANG Kai1, WANG Ruoding1, JIN Wenxing2. Power Information Data Fusion Model Based on Improved Extreme Learning Algorithm [J]. Computer and Modernization, 2024, 0(10): 61-64.
[12]	YANG Shijun1, DI Guangyi1, GAO Jun1, CHEN Jianfei1, WANG Yaokun1, JI Xiaohan2. Sentiment Consistency Detection Based on Cross Modal Attention Fusion and#br# Information Perception [J]. Computer and Modernization, 2024, 0(10): 113-119.
[13]	HOU Congying, YANG Wengqing, WANG Zhao, CHENG Cong. Speech Enhancement Based on Time-frequency Self-attention Residual Temporal#br# Convolutional Networks [J]. Computer and Modernization, 2024, 0(09): 20-24.
[14]	ZHANG Ze1, ZHANG Jianquan2, 3, ZHOU Guopeng2, 3. Camera Module Defect Detection Based on Improved YOLOv8s [J]. Computer and Modernization, 2024, 0(09): 107-113.
[15]	ZHENG Shangpo1, CHEN Defu1, LI Jianli2, LIN Guoxian2, WANG Xingping3. Pedestrian Tracking Algorithm Based on Improved YOLOv5s and DeepSORT [J]. Computer and Modernization, 2024, 0(08): 54-58.