Development and Evaluation of Low-Jitter Hand Tracking System for Improving Typing Efficiency in Virtual Reality Workspace

<title>Abstract</title><p>Virtual reality (VR) technology promises to transform immersive experiences across various applications, particularly within office environments. Despite its potential, the challenge of achieving efficient text entry in VR persists. This study addresses this obstacle by introducing a novel machine learning-based solution, namely the 2S-LSTM typing method, to enhance text entry performance in VR. The 2S-LSTM method utilizes the back-of-the-hand image, employing a two-stream Long Short-Term Memory (LSTM) network and a Kalman Filter (KF) to enhance hand position tracking accuracy and minimize jitter. Through statistical analysis of the data collected in the experiment and questionnaire results, we confirmed the effectiveness of the proposed method. In addition, we conducted an extra experiment to explore the differences in users’ typing behavior between regular typing and VR-based typing. This additional experiment provides valuable insights into how users adapt their typing behavior in different environments. These findings represent a significant step in advancing text entry within VR, setting the stage for immersive work experiences in office environments and beyond.</p>