Human factors modeling using machine learning for fusion energy applications

Abstract

This dissertation was aimed at investigating the challenges of cognitive fatigue and workload in remote handling operations within fusion reactor environments. An integrated multimodal framework was proposed, combining electroencephalography (EEG) and remote photoplethysmography (rPPG) technologies to facilitate real-time operator state monitoring and task-adaptive human–machine collaboration.

A low-density frontal EEG system was employed to detect cognitive fatigue through advanced spectral and nonlinear complexity analyses. To enhance classification performance under operational conditions, a lightweight Forward-Forward Graph Convolutional Network (F-FGCN) was introduced, which was demonstrated to exhibit strong robustness and computational efficiency. The model was shown to achieve an average classification accuracy of 82.21%, an F1-score of 82.63%, and an AUC of 0.876 in cross-subject validation.

In parallel, a non-contact rPPG-based physiological monitoring system was developed based on the ST-ASENet architecture. This model was demonstrated to reliably estimate heart rate (HR) and heart rate variability (HRV) from facial video and was shown to generalize strongly across diverse datasets, achieving an MAE of 2.88 and RMSE of 3.37 on BUAA, and a Pearson correlation coefficient of 0.86 on UBFC. These results confirmed its suitability for continuous online monitoring in dynamic operational settings.

The proposed framework was designed to incorporate both feature-level and decisionlevel fusion strategies, enabling the construction of a closed-loop cognitive monitoring system. This hybrid interface was configured to dynamically adjust interface complexity, task pacing, and autonomy levels in response to real-time cognitive state estimations. Simulation experiments were conducted, and significant performance gains were demonstrated, including improved classification accuracy, reduced false positives, and enhanced responsiveness in cognitively demanding scenarios.

Furthermore, a novel token-based representation learning paradigm was introduced. Multimodal signals—comprising EEG, PPG, and robotic interaction features—were encoded as semantic tokens and were processed using a frozen Transformer encoder. A modular regression head was subsequently employed to estimate event-level cognitive load by predicting fatigue and attention scores. The proposed method was shown to achieve a mean absolute error of 0.078 and a Pearson correlation of 0.87 on a 480-sample dataset, without the need for end-to-end fine-tuning.

Collectively, this work was demonstrated to provide a robust and scalable approach for continuous cognitive state monitoring in high-risk, high-load human–robot interaction environments such as fusion reactor maintenance and remote robotic operation, where accurate, real-time human state assessment was essential for safety and performance optimization.