Enhancing Interface Design with AI: An Exploratory Study on a ChatGPT-4-Based Tool for Cognitive Walkthrough Inspired Evaluations

This paper introduces CWGPT, a ChatGPT-4-based tool designed for Cognitive Walkthrough (CW) inspired evaluations of web interfaces. The primary goal is to assist users, particularly students and inexperienced designers, in evaluating web interfaces. Our tool, operating as a conversational agent, provides detailed evaluations of a user-specified task by intelligently guessing the subtasks and actions required to accomplish them, answering the standard CW questions, and providing helpful feedback and practical suggestions to improve the usability of the analyzed interface. For our study, we selected a group of web applications designed by students from a Web and Software Architecture course. We compare the outcome of the CWs we executed on ten web apps against the corresponding CWGPT analyses. We then describe the study we conducted involving five author-students to assess the tool's efficacy in helping them recognize and solve usability issues. In addition to introducing a novel adaptation of ChatGPT, the outcomes of the described experience underscore the promising potential of AI in usability evaluations.