Designing an in-browser knowledge base for a student tutor chat bot

Abstract

The integration of Large Language Models (LLMs) into educational environments offers significant potential for personalized tutoring. However, deploying commercial models introduces substantial hardware costs and data privacy concerns. This thesis addresses these challenges by proposing a novel, client-side Retrieval-Augmented Generation (RAG) architecture. The proposed system shifts the embedding generation and inference processes entirely to the user's web browser using WebLLM and Transfromers.js. This decentralized approach eliminates the need for expensive server infrastructure while ensuring that student data remains private.

We developed a functional prototype and evaluated it within a university programming course. The study investigated two primary research questions. First, we examined how advanced query transformation mitigates pedagogical limitations in naive RAG systems. Second, we assessed whether an in-browser Small Language Model (SLM) can perform as effectively as a commercial LLM for tutoring. The evaluation results demonstrated that the client-side RAG system successfully provides structured, step-by-step educational scaffolding. It effectively limits the scope of answers to prevent hallucinations. However, user preference strongly favored the direct problem-solving approach of the commercial baseline model over the system's pedagogical scaffolding. Furthermore, a significant disparity in reasoning capabilities between the local 3B parameter model and the commercial frontier model was observed.

In conclusion, the client-side RAG architecture presents a highly viable technical foundation for deploying privacy-preserving AI tutors. While current small language models require further advancements in instruction-following capabilities to match commercial counterparts, this architectural paradigm offers a cost-effective and scalable solution for educational institutions worldwide.