Enhancing handicraft intangible cultural heritage learning through immersive VR serious games: a multimodal study on natural interaction effectiveness

IntroductionWith the integration of natural interaction technologies such as gesture recognition and eye tracking into virtual reality (VR) systems, new possibilities have emerged for VR-based education of intangible cultural heritage (ICH) craftsmanship. Using Bai ethnic tie-dyeing as a case, this study developed an immersive virtual reality serious game (IVR-SG) grounded in Self-Determination Theory (SDT) to explore the differences between natural and controller-based interaction in VR learning of handicraft ICH.MethodsA total of 68 participants were randomly assigned to a natural interaction group or a controller interaction group. Learning processes were assessed through eye-tracking, electrodermal activity (EDA) and questionnaires.ResultsThe results revealed three key findings. First, participants using natural interaction demonstrated higher attentional focus during craft process simulation and faster command selection, but exhibited higher cognitive loads in precision tasks. Second, the natural interaction group reported significantly higher flow experience and self-efficacy. Third, the natural interaction group achieved significantly higher post-test knowledge scores, while both groups achieved comparable in-game procedural performance. Furthermore, regression analysis indicated that prior knowledge, flow experience, self-efficacy, and blink rates were positively correlated with knowledge acquisition.DiscussionThese findings highlight the differential impacts of interaction modalities in VR-based handicraft ICH education and provide design references for optimizing natural interaction in VR serious games for ICH.