人机交互中情感状态单多模的人类感知

doi:10.3969/j.issn.1006-2475.2016.10.022

计算机与现代化

人机交互中情感状态单多模的人类感知

河海大学计算机与信息学院,江苏南京211100

收稿日期:2016-03-11 出版日期:2016-10-15 发布日期:2016-10-14
作者简介:姜雪婷（1992-），女，江苏连云港人，河海大学计算机与信息学院硕士研究生，研究方向：人机交互；王宇（1979-），男，云南昆明人，副研究员，硕士生导师，博士，研究方向：云计算技术；田丽迎（1991-），女，硕士研究生，研究方向：云计算。
基金资助:
国家自然科学基金资助项目（61103017）

Unimodal and Multimodal Human Perception of Affective States #br# During Humancomputer Interaction

College of Computer and Information, Hohai University, Nanjing 211100, China

Received:2016-03-11 Online:2016-10-15 Published:2016-10-14

摘要/Abstract

摘要：

情感识别是人机交互领域的重要研究课题之一，随着研究通道的增多，研究成本和工作量也越来越大。本文在7种模式下通过未受训的观察员来检测人机交互过程中人类的基本情感状态（包括：厌

恶、恐惧、快乐、悲伤和惊讶）的自然表达，并评估稳定性。用混合效应逻辑回归模型对2个观察员（oo）之间的一致性进行计算和分析，结果显示一致性普遍偏低。除了比较单模态和多模态的整体一致

性，还比较了单个情感状态在单个模式下的一致性，而比较结果则用超可加性、可加性、冗余性和抑制性效应进行分类。目前，自动情感检测结果的意义还在研究中。

关键词: 人机交互, 情感检测, 多模态

Abstract:

Emotion perception is one of the most important research topics in the field of humancomputer interaction. With the increase of channels, research costs and the

workload are also increasing. In this paper, the human’s basic emotion states were been detected by untrained people in seven conditions during humancomputer interaction, and

assess stability. It computed and analyzed the agreement between two observers(oo) with mixedeffects logistic regression models. The result is generally low. In addition to

the overall consistency of the unimodal and multimodal condition, it also compared the consistency of individual affective states in a single model, and classified the results

with the superadditive, additive, redundancy and inhibitory effect. The significance of the results of automatic emotion detection is still been discussed.

Key words: humancomputer interaction, emotion perception, multimodal affect detection

中图分类号:

TP39

姜雪婷,王宇,田丽迎. 人机交互中情感状态单多模的人类感知[J]. 计算机与现代化, doi: 10.3969/j.issn.1006-2475.2016.10.022.

JIANG Xueting, WANG Yu, TIAN Liying. Unimodal and Multimodal Human Perception of Affective States #br# During Humancomputer Interaction[J]. Computer and Modernization, doi: 10.3969/j.issn.1006-2475.2016.10.022.

参考文献

［1］Calvo R A, D’Mello S K. Affect detection: An interdisciplinary review of models, methods, and their applications［J］. IEEE Transactions on Affective Computing, 2010,1

(1):18-37.
［2］黄力行,辛乐,赵礼悦,等. 自适应权重的双模态情感识别［J］. 清华大学学报(自然科学版), 2008(s1).
［3］Russell J A. Forcedchoice response format in the study of facial expression［J］. Motivation and Emotion, 1993,17(1):41-51.
［4］D’Mello Sidney K, Graesser A. Multimodal semiautomated affect detection from conversational cues, gross body language, and facial features［J］. User Modeling and User

Adapted Interaction, 2010.20(2):147-187.
［5］D’Mello Sidney K, Dowell Nia, Graesser A. Unimodal and multimodal human perception of naturalistic nonbasic affective states during humancomputer interactions［J］.

IEEE Transactions on Affective Computing, 2013,4(4):452-465.
［6］Aleix Martinez, Du Shichuan. A model of the perception of facial expressions of emotion by humans: Research overview and perspectives［J］. Journal of Machine Learning

Research, 2012,13(13):1589-1608.
［7］Camras L, Shutter J. Emotional facial expressions in infancy［J］. Emotion Rev., 2010,2(2):120-129.
［8］De Gelder B, Vanden Stock J. Real faces, real emotions: Perceiving facial expressions in naturalistic contexts of voices, bodies, and scenes［M］// Oxford Handbook of Face

Perception. Oxford University Press, 2011:535-550.

［9］D’Mello S K, Graesser A. Language and discourse are powerful signals of student emotions during tutoring［J］. IEEE Transactions on Learning Technologies, 2012,5(4):304-

317.
［10］D’Mello S K. Kory J. Consistent but modest: A metaanalysis on unimodal and multimodal affect detection accuracies from 30 studies［C］// Proc. 14th ACM Int’l Conf.

Multimodal Interaction. 2012:507-518.
［11］Stemmler G, Heldmann M， Pauls C A, et al. Constraints for emotion specificity in fear and anger: The context counts［J］. Psychophysiology, 2001,38(2):275-291.
［12］Lindquist K A, Barrett L F， Blissmoreau E, et al. Language and the perception of emotion［J］. Emotion, 2006,6(1):125-138
［13］Coan J A. Emergent ghosts of the emotion machine［J］. Emotion Rev., 2010,2(3):274-285.
［14］Frank M G, Stennett J. The forcedchoice paradigm and the perception of facial expressions of emotion［J］. J. Personality and Social Psychology, 2001,80(1):75-85.
［15］D’Mello S, Graesser A. Automatic detection of learner’s affect from gross body language［J］. Applied Artificial Intelligence, 2009,23(2):123-150,.
［16］Juslin P N, Laukka P. Communication of emotions in vocal expression and music performance: Different channels, same code?［J］. Psychological Bulletin, 2003,129(5):770-

814.

[1]	周义凯，王宇，赵勇飞，袁燕. 基于CNN的人体姿态识别[J]. 计算机与现代化, 2019, 0(02): 49-.
[2]	何坤金1，张莉军1，李强1，刘建新2. 基于OSG的航道船舶三维可视化信息管理系统[J]. 计算机与现代化, 2015, 0(11): 89-92.
[3]	顾绮芳. 上下肢康复训练器控制系统的设计与实现 [J]. 计算机与现代化, 2013, 1(11): 214-218.
[4]	高云丽;刘贤梅. 基于Flash的微生物发酵平台[J]. 计算机与现代化, 2013, 218(10): 51-54.
[5]	聂岩峰;田田;吴昊. 基于GIS指挥决策系统的多通道交互研究[J]. 计算机与现代化, 2013, 1(1): 203-206.
[6]	李钧;. 一种新型红外多点触摸识别算法[J]. 计算机与现代化, 2012, 1(9): 178-180,.
[7]	葛茜倩. 基于LPC2478的嵌入式人机互动终端[J]. 计算机与现代化, 2012, 1(200): 37-03.
[8]	顾杲;谈代林;李昌勇;武和雷. 基于双CPU结构的电能质量监测系统设计[J]. 计算机与现代化, 2012, 208(12): 222-225.
[9]	邱银国孙久运徐小芳魏萌. 基于Virtools的虚拟环境中视频应用技术研究 [J]. 计算机与现代化, 2011, 194(10): 54-56.

人机交互中情感状态单多模的人类感知

Unimodal and Multimodal Human Perception of Affective States #br# During Humancomputer Interaction

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

本文评价