Mitigation of hallucination in response to mental health counseling by large language models

Abstract

Large language models (LLMs) demonstrate hallucination behavior, which has been extensively documented in prior research. One key source of this issue is the mismatch between the knowledge acquired during pre-training and the information introduced during supervised fine-tuning. When models are exposed to unfamiliar or insufficiently represented knowledge in the fine-tuning stage, they may generate statements that lack factual grounding in their internalized knowledge. LLMs are being extensively investigated for application in mental health-related communication systems due to their robust conversational and reasoning abilities. These models demonstrated higher hallucination rate generating results easily but they are factually wrong or clinically inappropriate responses. So these responses provide considerable risks in mentally sensitive situations. In mental health applications, such errors might result in misleading information, enhanced anxiety, misconception of symptoms, or postponed professional assistance, underscoring the necessity for serious safety measures. To address this issue we have focused on reduction of hallucinations in mental health conversational system and introduced a hybrid neural-symbolic framework that integrates previous work on Selective Abstention Learning (SEAL) with a dynamically generated mental health knowledge network. The SEAL mechanism allows the model to explicitly decline to respond to queries that are dangerous, confusing, or lack credible evidence, while the knowledge graph offers organized, interpretable representations of symptom-disorder correlations to support model outputs. The knowledge graph is produced automatically and updated from reliable public medical sources and represented in RDF, eliminating the necessity for manual curation. Experimental evaluation demonstrates that the combined SEAL-KG approach reduces hallucinations, improves factual consistency, and yields safer responses during mental-health–related queries. This work contributes a technically robust and clinically cautious method for improving LLM behavior in mental-health support contexts.