When do users trust AI? : a study on the effects of visual design and tone in AI-based digital assistants

Abstract

A basic prerequisite to integrating AI-based digital assistants into academic support settings is trust, although there is little research to identify how conversational tone and visual interface design influence perceptions of trust. This paper examines these features in an experiment with 2x2 between subjects that used four chatbot prototypes with only a difference in tone (Friendly vs Formal) and visual design (Enhanced vs Minimal). Sixty participants chatted with a single chatbot and replied to a post-interaction questionnaire that measured trust, understandability, friendliness, willingness to rely on the assistant, reuse intention, and likelihood of recommendation. Results have demonstrated that Chatbot 1 (Friendly + Enhanced UI) scored the highest in all the trust indicators, such as friendliness (M=5.0), visual expressiveness (M=5.0), clarity (M=5.0), willingness to rely (M=4.44), reuse intention (M=4.85), and recommendation likelihood (M=4.85). Conversely, Chatbot 4 (Formal + Minimal UI) had the lowest average ratings of all with friendliness (M=2.5), visual expressiveness (M=3.0), and reuse intention (M=3.25). Formal chatbots achieved high marks on clarity and much lower marks on affective trust, proving that clarity is not enough to build user trust. Results also indicate a distinct interaction effect, with the positive effect of friendly tone being increased with better visual design whereas the perceived coldness of formal tone is increased with minimal interfaces. These findings validate that trust in conversational AI is not only determined by the accuracy of the content but also by the stylistic and visual modalities of information delivery. The research makes a contribution to the research in human-AI communication by validating the notion that trust of AI assistants can be influenced not only by the accuracy of the information but also by the tone and style under which the information is presented. The research offers an empirical clue on how to design more credible institutional chatbots. Practical implications emphasize that institutions should focus on tone and coherence of UI in chatbot design.