Deep Learning-Driven Behavioral Modeling in IoST for Mental Health Monitoring and Intervention

Abstract

Multimodal data have emerged as a cornerstone for understanding and analyzing complex human behaviors, particularly in mental health monitoring. In this study, we propose a deep learning-driven behavioral modeling framework for intelligence of social things (IoST)-based mental health monitoring and intervention, designed to integrate and analyze multimodal data—including text, speech, and physiological signals—captured from interconnected IoST devices. The framework incorporates an adaptive attention-based fusion mechanism that dynamically adjusts the contribution of each modality based on contextual relevance, enhancing the robustness of multimodal integration. Additionally, we employ a temporal-aware recurrent neural network with an attention mechanism to capture long-term dependencies and evolving behavioral patterns, ensuring precise mental health state prediction. To validate the framework, extensive experiments were conducted using three publicly available datasets: DAIC-WOZ, SEED, and MELD. Comparative experiments demonstrate the superior performance of the proposed framework, achieving state-of-the-art accuracy of 93.5%, F1-scores of 92.9%, and AUC-ROC of 0.95 values. Ablation studies highlight the critical roles of attention mechanisms and multimodal integration, showcasing significant performance improvements over single-modality and simplified fusion approaches. These findings underscore the framework’s potential as a reliable and efficient tool for real-time mental health monitoring in IoST environments, paving the way for scalable and personalized interventions.