Deep learning-based facial expression recognition and eye-tracking for detecting cognitive strain in university students

Abstract

Cognitive strain among university students is characterised by sustained mental overload and reduced capacity to process academic material. Current detection relies almost exclusively on self-report questionnaires, which miss real-time signals and are prone to response bias. A convolutional neural network was trained on the combined FER2013 and CK+ datasets, achieving 62.0% test accuracy on 7,360 samples, within the human-level range reported in the literature (65 ± 5%). The model was integrated into a webcam-based pipeline with dlib and MediaPipe Iris eye-tracking, producing a composite strain score from blink rate, PERCLOS, gaze jitter, iris diameter, and emotion classification, piloted with five students during a reading task.

The pipeline produced coherent strain scores across all five sessions, rising at reading onset and at the transition to comprehension questions. The eye-tracking channels behaved as expected, while the emotion channel showed a systematic bias toward misclassifying concentration as sadness. Strain scores partially corresponded with self-reported mental demand (r = 0.471), with stronger correspondence between comprehension performance and self-perceived success (r = 0.943). This thesis demonstrates the feasibility of a webcam-only cognitive strain detection system as a foundation for student wellbeing tools.