RAG ‘n’ Roll Industrial Management Game: Building and Testing a Large Language Model Assistant for Operations Education

Abstract

Simulation-based games are widely used in engineering and management studies to support decision-making in complex and dynamic environments. This study presents an exploratory case study of the design, implementation, and evaluation of a Retrieval-Augmented Generation (RAG) assistant that uses 10 years of instructor-answered student questions to provide context-aware support during gameplay. The RAG chatbot was designed and implemented to support facilitation by clarifying game mechanics, terminology, and previously addressed strategic considerations, helping teams navigate difficult decision-making processes without providing direct solutions. The assistant was implemented on a low-code, web-based artificial intelligence (AI) orchestration platform with an integrated vector database and tested during development. Perceived usefulness was evaluated through post-course student feedback across two independent markets with identical game mechanics. About 61% of students reported using the bot, while 39% did not. Analysis of survey data indicates students generally rated the chatbot as moderately useful, with extreme evaluations rare; at the same time, the overall semantic orientation of feedback was predominantly positive. Findings suggest that the chatbot functioned successfully as a contextual support mechanism rather than a direct driver of performance outcomes. This study demonstrates the feasibility of designing and evaluating an AI-enabled learning support tool in an industrial engineering and management course.