The influence of chatbot generated misinformation on student performance in mathematics : a study of trust and critical evaluation among high school students in Singapore

Abstract

LLM-powered chatbots such as ChatGPT, Gemini, LLaMA, and Claude have emerged as accessible and widely adopted platforms for personalised learning support, but they can also propagate misinformation. This Master’s thesis examined the influence of LLM chatbot-generated misinformation on student performance in Mathematics using data from a high school in Singapore. The study followed a mixed-method approach. Data was initially collected through a baseline assessment, which was a survey administered to understand the starting point of participants' understanding and usage patterns of chatbots and traditional search engines. This was followed by an intervention experiment, where participants were divided into two groups; the control and the treatment group. Participants interacted with two versions of a Modulus Operations Chatbot and solved a twenty-minute quiz. The participants were allowed to use a Modulus Operations Chatbot for assistance. The control group had a fully functional Modulus Operations Chatbot that explained modulus operations. The treatment group had a Modulus Operations Chatbot that explained modulus operations and also solved modulus-related questions. However, the Modulus Operations Chatbot of the treatment group was deliberately programmed to provide wrong answers to two out of the five questions on the quiz. Both participants in the control and treatment groups were allowed to complete the modulus operations quiz with their version of the Modulus Operations Chatbot. The experiments ended with a post-assessment, which was a survey administered to participants to understand how they rated their performance after interacting with the Modulus Operations Chatbot. Both the lab experiment and the online surveys were analysed using Stata to obtain the descriptive statistics, correlation and Ordinary Least Squares (OLS) Regression. The findings from the study confirm that students trust and often relied on the Modulus Operation Chatbots' responses without further analysis or verification, thus replicating multiple injected errors. The majority of the wrong answers by treatment group participants in the quiz were replicated injected errors. A high number of replicated injected errors is associated with a lower grade in the quiz. Participants who self-identified as Girls replicated more injected errors in the quiz than those who self-identified as boys. Participants within the treatment group who trust LLM-based chatbot services more than Wikipedia or Google replicated less injected errors than those who trusted Google or Wikipedia more than LLM-based chatbot services. The study recommends that students and teachers be informed about the limitations of LLM Chatbot services and how LLM Chatbot services can generate incorrect responses. Additionally, companies should test their LLMs more thoroughly to catch bugs in their models, as this will make their models produce more accurate and correct responses to user queries.