Investigating Perceived Reliability of ChatGPT Outputs through Design Change & Educational Interventions

Generative AI chatbots like ChatGPT can be unreliable. Despite this, ChatGPT has become widely used in higher education settings. Little research has explored ways to address the concern of AI plagiarism stemming from ChatGPT’s growing use in universities. Applying Ajzen’s (1991) Theory of Planned Behaviour, the current project aimed to address this gap by conducting two exploratory studies that attempted to change student attitudes toward the reliability of ChatGPT outputs. Study 1 focused on whether applying Wickens’ Display Design principles as a design change intervention would decrease people’s perceived reliability of original, congruent, and incongruent excerpts. Study 2 focused on whether SACCIA (Sufficient, Accurate, Clear, Contextualised, and Interpersonally Adapted) adherent seminar- and tutorial-based education on the unreliability of generative AI would decrease attitudes of perceived reliability toward ChatGPT outputs. While most findings did not show significant differences, many were trending toward what was expected. Consequently, the design and educational intervention provided insight into what factors may influence student attitudes toward the reliability of ChatGPT outputs. These combined findings could be applied to 100- and 200-level psychology laboratory classes. However, future research is required to determine whether the findings of this project can be combined to effectively educate students about the unreliability of generative AI.