Characterizing Physiological Responses to Fear, Frustration, and Insight in Virtual Reality

Tiffany Luong and Christian Holz.

IEEE Transactions on Visualization and Computer Graphics 2022.

Teaser

Physiological Responses to Fear, Frustration, and Insight teaser image

In a study with 24 participants, we elicited transient psychological states using: (a&b) moments of fear, (c&d) moments of frustration when accomplishing a task, and (e) moments of insight when solving a puzzle. During the study, we measured participants’ cardiac, pulmonary, electrodermal, and pupillary activity to capture participants’ physiological behavior and present an analysis to understand the short-term responses to events in VR.

Publication

Tiffany Luong and Christian Holz. Characterizing Physiological Responses to Fear, Frustration, and Insight in Virtual Reality. In IEEE Transactions on Visualization and Computer Graphics 2022.

Abstract

Physiological sensing often complements studies of human behavior in virtual reality (VR) to detect users’ affective and cognitive states. Some psychological states, such as fear and frustration, can be particularly hard to differentiate from a physiological perspective as they are close in the arousal and valence emotional space. Moreover, it is largely unclear how users’ physiological reactions are expressed in response to transient psychological states such as fear, frustration, and insight—especially since these are rich indicators for characterizing users’ responses to dynamic systems but are hard to capture in highly interactive settings. We conducted a study (N=24) to analyze participants’ pulmonary, electrodermal, cardiac, and pupillary responses to moments of fear, frustration, and insight in immersive settings. Participants interacted in five VR environments, throughout which we measured their physiological reactions and analyzed the patterns. We also measured subjective fear and frustration using questionnaires. We found differences between fear and frustration pupillary, respiratory, and electrodermal responses, as well as between the pupillary changes that followed fear in a horror game and those that followed fear in a vertigo experiment. We present the relationships between fear levels, frustration levels, and their physiological responses. To detect these affective events and states, we introduce user-independent binary classification models that achieved an average micro F1 score of 71% for detecting fear in a horror game, 75% for fear of vertigo, 76% for frustration, and 75% for insight, showing the promise for detecting these states from passive and objective signals.