Towards Evaluating Visually Induced Motion Sickness Using Facial Expressions

<p>As remote control technology for hydraulic excavators continues to develop, the issue of visually induced motion sickness (VIMS) during operation has become a concern. One method of detecting VIMS is through biometric measurement, but the use of sensors can be burdensome, particularly for remote operation.</p><p>Therefore, this study focused on non-invasive facial analysis using an RGB camera and investigated changes in facial expressions during VIMS. With the participation of 11 healthy male subjects experienced in hydraulic excavator work, VIMS was subjectively evaluated on a five-point scale during simulated excavation work, and the Simulation Sickness Questionnaire (SSQ) was used to evaluate the subjects before and after the experiment. Facial expressions were analyzed using Google's Mediapipe FaceMesh, which extracted 468 facial feature points from the subjects' facial images during the work. Changes in facial feature points were compared with the subjective evaluation of VIMS.</p><p>Based on the SSQ, six participants experienced VIMS, and all of them showed changes in feature point coordinates near their cheeks. Additionally, for two participants, characteristic movements were observed near the nasolabial folds, where feature point coordinates moved away from the center of the face. This suggests that VIMS can be detected through changes in facial expressions.</p><p>In the future, we will collect more data for confirmation, and by combining this with instantaneous facial expression changes due to emotional changes, we may be able to detect negative facial expressions due to VIMS discomfort by combining it with the momentary facial expression changes caused by emotional changes. It is also necessary to distinguish negative facial expressions due to VIMS from those due to fatigue or decreased arousal caused by work.</p>