LoRA-Based Cascaded Multimodal Fusion for Action Recognition in Medical Training Environments
Article excerpt
This paper presents a cascaded Low-Rank Adaptation (LoRA)-based multimodal fusion framework for action and activity recognition in healthcare-oriented training environments. The proposed architecture combines parameter-efficient modality-specific adaptation with sequential fusion, enabling modalities to be integrated in stages without retraining previously…
IntroductionPilot performance is influenced by cognitive workload, operational stress, and psychophysiological adaptation during demanding flight tasks. This pilot feasibility study evaluated the applicability of multimodal workload assessment during simulator-based instrument flight operations and explored differences in workload response and operational performance between pilot trainees at different stages of training.MethodsTwenty-five students from the University of Defence completed a standardized simulator scenario involving an ILS instrument approach under instrument meteorological conditions. Subjective workload was assessed using the NASA Task Load Index (NASA-TLX), physiological response was evaluated using continuous heart-rate monitoring, and operational performance was analyzed using simulator-derived flight parameters. Composite Performance Deviation Index (PDI) and Workload Index (WI) values were subsequently calculated.ResultsThe results revealed substantial inter-individual variability in subjective workload, physiological activation, and operational performance. Considerable overlap was observed between year-of-study groups across most evaluated indicators. Several participants maintained relatively stable flight performance despite elevated physiological activation or subjective workload, suggesting that workload response was not uniformly reflected in operational performance measures. The integrated WI-PDI framework additionally provided an exploratory visualization of workload-performance relationships at the individual-participant level.DiscussionThe study demonstrated the practical feasibility of implementing multimodal workload assessment in a simulator-based flight-training environment. The findings support the use of integrated psychophysiological monitoring for future research on workload adaptation and operational performance during pilot training. However, the proposed WI and PDI metrics should be regarded as exploratory measures requiring further validation in larger pilot populations.